Kullback-Leibler Divergence-Based Regularized Normalization for Low-Resource Tasks

Large pretrained models, like BERT, GPT, and Wav2Vec, have demonstrated their ability to learn transferable representations for various downstream tasks. However, obtaining a substantial amount of supervised data remains a challenge due to resource and time limitations. As a solution, researchers ha...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	IEEE transactions on artificial intelligence 2024-06, Vol.5 (6), p.2638-2650
Hauptverfasser:	Kumar, Neeraj, Narang, Ankur, Lall, Brejesh
Format:	Artikel
Sprache:	eng
Schlagworte:	Adaptation models Artificial intelligence Data models Kullback–Leibler (KL) regularization large pretrained model low-resource tasks Mathematical models normalization Task analysis Training Tuning
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!