Accelerated multi-neural network training method based on programmable network design

The invention relates to an accelerated multi-neural network training method based on programmable network design. According to the method, a programmable switch is used as a parameter server, and streaming aggregation is carried out by using an intra-network aggregation technology; in a deployment...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: HUANG KAI, LIN SHENGRUI, CHEN XINYANG, ZHU LONGLONG, LIU HONGYAN, ZHANG DONG, ZHOU JIASHENG, SHI PENGPAI, ZHENG LINYING
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention relates to an accelerated multi-neural network training method based on programmable network design. According to the method, a programmable switch is used as a parameter server, and streaming aggregation is carried out by using an intra-network aggregation technology; in a deployment stage of neural network training, limitation and resource difference of a real physical network are considered, selection of a parameter server and a Worker node is optimized through an ILP and a greedy algorithm, so that training efficiency and system performance are improved, and dynamic optimization of a deployment scheme can be supported in a training process; meanwhile, a new data packet header field protocol is designed to support simultaneous online training of multiple neural networks, and each job and data packet priority are endowed to improve system coordination. According to the method, the network flexibility, performance and efficiency are effectively improved by optimizing selection of aggregation no