Gradual Learning: Optimizing Fine-Tuning with Partially Mastered Knowledge in Large Language Models

During the pretraining phase, large language models (LLMs) acquire vast amounts of knowledge from extensive text corpora. Nevertheless, in later stages such as fine-tuning and inference, the model may encounter knowledge not covered in the initial training, which can lead to hallucinations and degra...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2024-10
Hauptverfasser:	Li, Bozhou, Liang, Hao, Yang, Li, Fu, Fangcheng, Yin, Hongzhi, He, Conghui, Zhang, Wentao
Format:	Artikel
Sprache:	eng
Schlagworte:	Accuracy Datasets Knowledge Knowledge acquisition Large language models Performance degradation
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!