Transfer Learning for Text Diffusion Models

In this report, we explore the potential for text diffusion to replace autoregressive (AR) decoding for the training and deployment of large language models (LLMs). We are particularly interested to see whether pretrained AR models can be transformed into text diffusion models through a lightweight...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2024-01
Hauptverfasser:	Han, Kehang, Kenealy, Kathleen, Barua, Aditya, Fiedel, Noah, Constant, Noah
Format:	Artikel
Sprache:	eng
Schlagworte:	Autoregressive processes Decoding Diffusion rate Large language models Machine translation Quality assurance Training
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!