Scalable Fully Pipelined Hardware Architecture for In-Network Aggregated AllReduce Communication

The Ring-AllReduce framework is currently the most popular solution to deploy industry-level distributed machine learning tasks. However, only about half of the maximum bandwidth can be achieved in the optimal condition. In recent years, several in-network aggregation frameworks have been proposed t...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on circuits and systems. I, Regular papers Regular papers, 2021-10, Vol.68 (10), p.4194-4206
Hauptverfasser:	Liu, Yao, Zhang, Junyi, Liu, Shuo, Wang, Qiaoling, Dai, Wangchen, Cheung, Ray Chak Chung
Format:	Artikel
Sprache:	eng
Schlagworte:	Agglomeration AllReduce Bandwidth Bandwidths Cognitive tasks Collective communication Communication Computation distributed machine learning Hardware in-network aggregation Machine learning Network adapters Network topology Peer-to-peer computing Switches Task analysis Topology Training
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Schreiben Sie den ersten Kommentar!