Transformers Learn Shortcuts to Automata

Algorithmic reasoning requires capabilities which are most naturally understood through recurrent models of computation, like the Turing machine. However, Transformer models, while lacking recurrence, are able to perform such reasoning using far fewer layers than the number of reasoning steps. This...

Ausführliche Beschreibung

Bibliographische Detailangaben

Hauptverfasser:	Liu, Bingbin, Ash, Jordan T, Goel, Surbhi, Krishnamurthy, Akshay, Zhang, Cyril
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer Science - Formal Languages and Automata Theory Computer Science - Learning Statistics - Machine Learning
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!