More Room for Language: Investigating the Effect of Retrieval on Language Models

Retrieval-augmented language models pose a promising alternative to standard language modeling. During pretraining, these models search in a corpus of documents for contextually relevant information that could aid the language modeling objective. We introduce an 'ideal retrieval' methodolo...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2024-04
Hauptverfasser:	Samuel, David, Lucas Georges Gabriel Charpentier, Wold, Sondre
Format:	Artikel
Sprache:	eng
Schlagworte:	Context Controllability Modelling Retrieval
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!