STrack: A Reliable Multipath Transport for AI/ML Clusters
Emerging artificial intelligence (AI) and machine learning (ML) workloads present new challenges of managing the collective communication used in distributed training across hundreds or even thousands of GPUs. This paper presents STrack, a novel hardware-offloaded reliable transport protocol aimed a...
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | |
| container_volume | |
| creator | Le, Yanfang Pan, Rong Newman, Peter Blendin, Jeremias Kabbani, Abdul Jain, Vipin Sivaramu, Raghava Matus, Francis |
| description | Emerging artificial intelligence (AI) and machine learning (ML) workloads
present new challenges of managing the collective communication used in
distributed training across hundreds or even thousands of GPUs. This paper
presents STrack, a novel hardware-offloaded reliable transport protocol aimed
at improving the performance of AI /ML workloads by rethinking key aspects of
the transport layer. STrack optimizes congestion control and load balancing in
tandem: it incorporates an adaptive load balancing algorithm leveraging ECN,
while adopts RTT as multi-bit congestion indicators for precise congestion
window adjustment. Additionally, STrack facilitates out-of-order delivery,
selective retransmission, and swift loss recovery in hardware for multipath
environment. The extensive simulation comparing STrack and RoCEv2 demonstrates
that STrack outperforms RoCEv2 by up to 6X with synthetic workloads and by
27.4% with collective workloads, even with the optimized RoCEv2 system setup. |
| doi_str_mv | 10.48550/arxiv.2407.15266 |
| format | Article |
| fullrecord | <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2407_15266</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2407_15266</sourcerecordid><originalsourceid>FETCH-arxiv_primary_2407_152663</originalsourceid><addsrcrecordid>eNpjYJA0NNAzsTA1NdBPLKrILNMzMjEw1zM0NTIz42SwDA4pSkzOtlJwVAhKzclMTMpJVfAtzSnJLEgsyVAAyuUVF-QXlSik5RcpOHrq-_ooOOeUFpekFhXzMLCmJeYUp_JCaW4GeTfXEGcPXbAl8QVFmbmJRZXxIMviwZYZE1YBAMuDMwY</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>STrack: A Reliable Multipath Transport for AI/ML Clusters</title><source>arXiv.org</source><creator>Le, Yanfang ; Pan, Rong ; Newman, Peter ; Blendin, Jeremias ; Kabbani, Abdul ; Jain, Vipin ; Sivaramu, Raghava ; Matus, Francis</creator><creatorcontrib>Le, Yanfang ; Pan, Rong ; Newman, Peter ; Blendin, Jeremias ; Kabbani, Abdul ; Jain, Vipin ; Sivaramu, Raghava ; Matus, Francis</creatorcontrib><description>Emerging artificial intelligence (AI) and machine learning (ML) workloads
present new challenges of managing the collective communication used in
distributed training across hundreds or even thousands of GPUs. This paper
presents STrack, a novel hardware-offloaded reliable transport protocol aimed
at improving the performance of AI /ML workloads by rethinking key aspects of
the transport layer. STrack optimizes congestion control and load balancing in
tandem: it incorporates an adaptive load balancing algorithm leveraging ECN,
while adopts RTT as multi-bit congestion indicators for precise congestion
window adjustment. Additionally, STrack facilitates out-of-order delivery,
selective retransmission, and swift loss recovery in hardware for multipath
environment. The extensive simulation comparing STrack and RoCEv2 demonstrates
that STrack outperforms RoCEv2 by up to 6X with synthetic workloads and by
27.4% with collective workloads, even with the optimized RoCEv2 system setup.</description><identifier>DOI: 10.48550/arxiv.2407.15266</identifier><language>eng</language><subject>Computer Science - Networking and Internet Architecture</subject><creationdate>2024-07</creationdate><rights>http://creativecommons.org/licenses/by/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2407.15266$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2407.15266$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Le, Yanfang</creatorcontrib><creatorcontrib>Pan, Rong</creatorcontrib><creatorcontrib>Newman, Peter</creatorcontrib><creatorcontrib>Blendin, Jeremias</creatorcontrib><creatorcontrib>Kabbani, Abdul</creatorcontrib><creatorcontrib>Jain, Vipin</creatorcontrib><creatorcontrib>Sivaramu, Raghava</creatorcontrib><creatorcontrib>Matus, Francis</creatorcontrib><title>STrack: A Reliable Multipath Transport for AI/ML Clusters</title><description>Emerging artificial intelligence (AI) and machine learning (ML) workloads
present new challenges of managing the collective communication used in
distributed training across hundreds or even thousands of GPUs. This paper
presents STrack, a novel hardware-offloaded reliable transport protocol aimed
at improving the performance of AI /ML workloads by rethinking key aspects of
the transport layer. STrack optimizes congestion control and load balancing in
tandem: it incorporates an adaptive load balancing algorithm leveraging ECN,
while adopts RTT as multi-bit congestion indicators for precise congestion
window adjustment. Additionally, STrack facilitates out-of-order delivery,
selective retransmission, and swift loss recovery in hardware for multipath
environment. The extensive simulation comparing STrack and RoCEv2 demonstrates
that STrack outperforms RoCEv2 by up to 6X with synthetic workloads and by
27.4% with collective workloads, even with the optimized RoCEv2 system setup.</description><subject>Computer Science - Networking and Internet Architecture</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNpjYJA0NNAzsTA1NdBPLKrILNMzMjEw1zM0NTIz42SwDA4pSkzOtlJwVAhKzclMTMpJVfAtzSnJLEgsyVAAyuUVF-QXlSik5RcpOHrq-_ooOOeUFpekFhXzMLCmJeYUp_JCaW4GeTfXEGcPXbAl8QVFmbmJRZXxIMviwZYZE1YBAMuDMwY</recordid><startdate>20240721</startdate><enddate>20240721</enddate><creator>Le, Yanfang</creator><creator>Pan, Rong</creator><creator>Newman, Peter</creator><creator>Blendin, Jeremias</creator><creator>Kabbani, Abdul</creator><creator>Jain, Vipin</creator><creator>Sivaramu, Raghava</creator><creator>Matus, Francis</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20240721</creationdate><title>STrack: A Reliable Multipath Transport for AI/ML Clusters</title><author>Le, Yanfang ; Pan, Rong ; Newman, Peter ; Blendin, Jeremias ; Kabbani, Abdul ; Jain, Vipin ; Sivaramu, Raghava ; Matus, Francis</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-arxiv_primary_2407_152663</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Computer Science - Networking and Internet Architecture</topic><toplevel>online_resources</toplevel><creatorcontrib>Le, Yanfang</creatorcontrib><creatorcontrib>Pan, Rong</creatorcontrib><creatorcontrib>Newman, Peter</creatorcontrib><creatorcontrib>Blendin, Jeremias</creatorcontrib><creatorcontrib>Kabbani, Abdul</creatorcontrib><creatorcontrib>Jain, Vipin</creatorcontrib><creatorcontrib>Sivaramu, Raghava</creatorcontrib><creatorcontrib>Matus, Francis</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Le, Yanfang</au><au>Pan, Rong</au><au>Newman, Peter</au><au>Blendin, Jeremias</au><au>Kabbani, Abdul</au><au>Jain, Vipin</au><au>Sivaramu, Raghava</au><au>Matus, Francis</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>STrack: A Reliable Multipath Transport for AI/ML Clusters</atitle><date>2024-07-21</date><risdate>2024</risdate><abstract>Emerging artificial intelligence (AI) and machine learning (ML) workloads
present new challenges of managing the collective communication used in
distributed training across hundreds or even thousands of GPUs. This paper
presents STrack, a novel hardware-offloaded reliable transport protocol aimed
at improving the performance of AI /ML workloads by rethinking key aspects of
the transport layer. STrack optimizes congestion control and load balancing in
tandem: it incorporates an adaptive load balancing algorithm leveraging ECN,
while adopts RTT as multi-bit congestion indicators for precise congestion
window adjustment. Additionally, STrack facilitates out-of-order delivery,
selective retransmission, and swift loss recovery in hardware for multipath
environment. The extensive simulation comparing STrack and RoCEv2 demonstrates
that STrack outperforms RoCEv2 by up to 6X with synthetic workloads and by
27.4% with collective workloads, even with the optimized RoCEv2 system setup.</abstract><doi>10.48550/arxiv.2407.15266</doi><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | DOI: 10.48550/arxiv.2407.15266 |
| ispartof | |
| issn | |
| language | eng |
| recordid | cdi_arxiv_primary_2407_15266 |
| source | arXiv.org |
| subjects | Computer Science - Networking and Internet Architecture |
| title | STrack: A Reliable Multipath Transport for AI/ML Clusters |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-23T06%3A01%3A51IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=STrack:%20A%20Reliable%20Multipath%20Transport%20for%20AI/ML%20Clusters&rft.au=Le,%20Yanfang&rft.date=2024-07-21&rft_id=info:doi/10.48550/arxiv.2407.15266&rft_dat=%3Carxiv_GOX%3E2407_15266%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |