Large Vocabulary Size Improves Large Language Models

This paper empirically investigates the relationship between subword vocabulary size and the performance of large language models (LLMs) to provide insights on how to define the vocabulary size. Experimental results show that larger vocabulary sizes lead to better performance in LLMs. Moreover, we c...

Ausführliche Beschreibung

Bibliographische Detailangaben

Hauptverfasser:	Takase, Sho, Ri, Ryokan, Kiyono, Shun, Kato, Takuya
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer Science - Computation and Language
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!