Prune Once for All: Sparse Pre-Trained Language Models

Transformer-based language models are applied to a wide range of applications in natural language processing. However, they are inefficient and difficult to deploy. In recent years, many compression algorithms have been proposed to increase the implementation efficiency of large Transformer-based mo...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2021-11
Hauptverfasser:	Zafrir, Ofir, Larey, Ariel, Boudoukh, Guy, Shen, Haihao, Wasserblat, Moshe
Format:	Artikel
Sprache:	eng
Schlagworte:	Accuracy Algorithms Coders Compression ratio Distillation Knowledge management Language Natural language Natural language processing Training
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!