Memory Augmented Language Models through Mixture of Word Experts

Scaling up the number of parameters of language models has proven to be an effective approach to improve performance. For dense models, increasing model size proportionally increases the model's computation footprint. In this work, we seek to aggressively decouple learning capacity and FLOPs th...

Ausführliche Beschreibung

Bibliographische Detailangaben

Hauptverfasser:	Santos, Cicero Nogueira dos, Lee-Thorp, James, Noble, Isaac, Chang, Chung-Ching, Uthus, David
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer Science - Computation and Language
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!