Training giant neural networks using pipeline parallelism

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training giant neural networks. One of the methods includes obtaining data specifying a partitioning of the neural network into N composite layers that form a sequence of composite layers, wherein eac...

Ausführliche Beschreibung

Bibliographische Detailangaben

Hauptverfasser:	Ngiam, Jiquan, Huang, Yanping, Cheng, Youlong, Lee, HyoukJoong, Chen, Dehao, Chen, Zhifeng
Format:	Patent
Sprache:	eng
Schlagworte:	CALCULATING COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS COMPUTING COUNTING PHYSICS
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!