Towards Anytime Fine-tuning: Continually Pre-trained Language Models with Hypernetwork Prompt

Continual pre-training has been urgent for adapting a pre-trained model to a multitude of domains and tasks in the fast-evolving world. In practice, a continually pre-trained model is expected to demonstrate not only greater capacity when fine-tuned on pre-trained domains but also a non-decreasing p...

Ausführliche Beschreibung

Bibliographische Detailangaben

Hauptverfasser:	Jiang, Gangwei, Jiang, Caigao, Xue, Siqiao, Zhang, James Y, Zhou, Jun, Lian, Defu, Wei, Ying
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer Science - Artificial Intelligence Computer Science - Computation and Language Computer Science - Learning
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!