ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning

We propose a new paradigm to continually evolve pretrained models, denoted ColD Fusion. It provides the benefits of multitask learning but leverages distributed computation with limited communication and eliminates the need for shared data. Consequentially, ColD Fusion can give rise to a synergistic...

Ausführliche Beschreibung

Bibliographische Detailangaben

Hauptverfasser:	Don-Yehiya, Shachar, Venezian, Elad, Raffel, Colin, Slonim, Noam, Katz, Yoav, Choshen, Leshem
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer Science - Computation and Language Computer Science - Distributed, Parallel, and Cluster Computing Computer Science - Learning
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!