Evaluating Transformer's Ability to Learn Mildly Context-Sensitive Languages

Despite the fact that Transformers perform well in NLP tasks, recent studies suggest that self-attention is theoretically limited in learning even some regular and context-free languages. These findings motivated us to think about their implications in modeling natural language, which is hypothesize...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2023-10
Hauptverfasser:	Wang, Shunjie, Steinert-Threlkeld, Shane
Format:	Artikel
Sprache:	eng
Schlagworte:	Context free languages Transformers
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!