How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad

Can Transformers predict new syllogisms by composing established ones? More generally, what type of targets can be learned by such models from scratch? Recent works show that Transformers can be Turing-complete in terms of expressivity, but this does not address the learnability objective. This pape...

Ausführliche Beschreibung

Bibliographische Detailangaben

Hauptverfasser:	Abbe, Emmanuel, Bengio, Samy, Lotfi, Aryo, Sandon, Colin, Saremi, Omid
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer Science - Artificial Intelligence Computer Science - Learning Statistics - Machine Learning
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!