v Pipe : A Virtualized Acceleration System for Achieving Efficient and Scalable Pipeline Parallel DNN Training

The increasing computational complexity of DNNs achieved unprecedented successes in various areas such as machine vision and natural language processing (NLP), e.g., the recent advanced Transformer has billions of parameters. However, as large-scale DNNs significantly exceed GPU’s physical memory li...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	IEEE transactions on parallel and distributed systems 2022-01, Vol.33 (3), p.489
Hauptverfasser:	Zhao, Shixiong, Li, Fanxin, Chen, Xusheng, Guan, Xiuxian, Jiang, Jianyu, Huang, Dong, Yuhao Qing, Wang, Sen, Wang, Peng, Zhang, Gong, Cheng, Li, Luo, Ping, Cui, Heming
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer architecture Graphics processing units Machine vision Memory management Natural language processing Partitioning Search algorithms Training Transformers
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!