A Catalog of Transformer Models

In the past few years we have seen the meteoric appearance of dozens of models of the Transformer family. This ORKG comparison contrasts models from different families including GPT, T5, LLaMA etc. on some salient aspects around their innovation and difference. This comparison is inspired after the paper "Transformer models: an introduction and catalog" by Xavier Amatriain, while seeking to be an extendable list of both properties and new contributions in terms of transformer models as they continue to be introduced.