mSLAM: Massively multilingual joint pre-training for speech and text

We present mSLAM, a multilingual Speech and LAnguage Model that learns cross-lingual cross-modal representations of speech and text by pre-training jointly on large amounts of unlabeled speech and text in multiple languages. mSLAM combines w2v-BERT pre-training on speech with SpanBERT pre-training o...

Ausführliche Beschreibung

Bibliographische Detailangaben

Hauptverfasser:	Bapna, Ankur, Cherry, Colin, Zhang, Yu, Jia, Ye, Johnson, Melvin, Cheng, Yong, Khanuja, Simran, Riesa, Jason, Conneau, Alexis
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer Science - Computation and Language Computer Science - Learning
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!