Using Selective Masking as a Bridge between Pre-training and Fine-tuning

Pre-training a language model and then fine-tuning it for downstream tasks has demonstrated state-of-the-art results for various NLP tasks. Pre-training is usually independent of the downstream task, and previous works have shown that this pre-training alone might not be sufficient to capture the ta...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2022-11
Hauptverfasser:	Lad, Tanish, Maheshwari, Himanshu, Kottukkal, Shreyas, Mamidi, Radhika
Format:	Artikel
Sprache:	eng
Schlagworte:	Downstream effects Masking Training
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!