A Data-Driven Packet Routing Algorithm for an Unmanned Aerial Vehicle Swarm: A Multi-Agent Reinforcement Learning Approach
Routing decisions made by unmanned aerial vehicle (UAV) swarms are affected by complex and dynamically changing topologies. A centralized routing algorithm imposes the entire computational burden on one module, and the high data dimensionality renders computation burdensome. In this letter, we devel...
Gespeichert in:
| Veröffentlicht in: | IEEE wireless communications letters 2022-10, Vol.11 (10), p.2160-2164 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 2164 |
|---|---|
| container_issue | 10 |
| container_start_page | 2160 |
| container_title | IEEE wireless communications letters |
| container_volume | 11 |
| creator | Qiu, Xiulin Xu, Lei Wang, Ping Yang, Yuwang Liao, Zhenqiang |
| description | Routing decisions made by unmanned aerial vehicle (UAV) swarms are affected by complex and dynamically changing topologies. A centralized routing algorithm imposes the entire computational burden on one module, and the high data dimensionality renders computation burdensome. In this letter, we develop a multi-agent reinforcement learning-based routing algorithm for a UAV swarm. The UAVs are trained in a data-driven manner to make distributed routing decisions. Factors that include channel quality, UAV movement, UAV overhead, and the extent of neighbor variation are incorporated into link quality assessment. Long short-term memory is used to improve the Actor and Critic networks, and more information on temporal continuity is added to facilitate adaptation to the dynamically changing environment. Simulations show that the proposed routing algorithm reduces data transmission delay and enhances the transmission rate compared with traditional routing algorithms. |
| doi_str_mv | 10.1109/LWC.2022.3195963 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_proquest_journals_2722546772</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9849115</ieee_id><sourcerecordid>2722546772</sourcerecordid><originalsourceid>FETCH-LOGICAL-c291t-d00ec2190efa0f0df91bc386ef0a785eab795bfc662ffd8e5ce0d58b240b9ac53</originalsourceid><addsrcrecordid>eNo9kM9LwzAUx4soOHR3wUvAc2eSNm3jrWz-goqiTo8hTV-2zDadaavoX2_mxt7lvQffH_AJgjOCJ4Rgflm8TycUUzqJCGc8iQ6CESUJDWkUs8P9HaXHwbjrVthPggkl2Sj4zdFM9jKcOfMFFj1J9QE9em6H3tgFyutF60y_bJBuHZIWzW0jrYUK5eCMrNEbLI2qAb18S9dcoRw9DHVvwnwB1qeAsd6noNl8BUhn_0PXa9dKtTwNjrSsOxjv9kkwv7l-nd6FxePt_TQvQkU56cMKY1CUcAxaYo0rzUmpoiwBjWWaMZBlylmpVZJQrasMmAJcsaykMS65VCw6CS62ub72c4CuF6t2cNZXCppSyuIkTalX4a1KubbrHGixdqaR7kcQLDaQhYcsNpDFDrK3nG8tBgD2cp7FnBAW_QHln3jN</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2722546772</pqid></control><display><type>article</type><title>A Data-Driven Packet Routing Algorithm for an Unmanned Aerial Vehicle Swarm: A Multi-Agent Reinforcement Learning Approach</title><source>IEEE Electronic Library (IEL)</source><creator>Qiu, Xiulin ; Xu, Lei ; Wang, Ping ; Yang, Yuwang ; Liao, Zhenqiang</creator><creatorcontrib>Qiu, Xiulin ; Xu, Lei ; Wang, Ping ; Yang, Yuwang ; Liao, Zhenqiang</creatorcontrib><description>Routing decisions made by unmanned aerial vehicle (UAV) swarms are affected by complex and dynamically changing topologies. A centralized routing algorithm imposes the entire computational burden on one module, and the high data dimensionality renders computation burdensome. In this letter, we develop a multi-agent reinforcement learning-based routing algorithm for a UAV swarm. The UAVs are trained in a data-driven manner to make distributed routing decisions. Factors that include channel quality, UAV movement, UAV overhead, and the extent of neighbor variation are incorporated into link quality assessment. Long short-term memory is used to improve the Actor and Critic networks, and more information on temporal continuity is added to facilitate adaptation to the dynamically changing environment. Simulations show that the proposed routing algorithm reduces data transmission delay and enhances the transmission rate compared with traditional routing algorithms.</description><identifier>ISSN: 2162-2337</identifier><identifier>EISSN: 2162-2345</identifier><identifier>DOI: 10.1109/LWC.2022.3195963</identifier><identifier>CODEN: IWCLAF</identifier><language>eng</language><publisher>Piscataway: IEEE</publisher><subject>Algorithms ; Artificial intelligence ; Autonomous aerial vehicles ; Changing environments ; Data transmission ; data-driven routing ; Decisions ; Heuristic algorithms ; link quality assessment ; LSTM ; Machine learning ; multi-agent reinforcement learning ; Multiagent systems ; Quality assessment ; Route planning ; Routing ; Signal to noise ratio ; Topology ; Transmission rate (communications) ; UAV swarm network ; Unmanned aerial vehicles ; Vehicle dynamics</subject><ispartof>IEEE wireless communications letters, 2022-10, Vol.11 (10), p.2160-2164</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c291t-d00ec2190efa0f0df91bc386ef0a785eab795bfc662ffd8e5ce0d58b240b9ac53</citedby><cites>FETCH-LOGICAL-c291t-d00ec2190efa0f0df91bc386ef0a785eab795bfc662ffd8e5ce0d58b240b9ac53</cites><orcidid>0000-0002-6636-8831 ; 0000-0002-9306-5844 ; 0000-0002-8535-6238 ; 0000-0002-1599-5480</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9849115$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,776,780,792,27901,27902,54733</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/9849115$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Qiu, Xiulin</creatorcontrib><creatorcontrib>Xu, Lei</creatorcontrib><creatorcontrib>Wang, Ping</creatorcontrib><creatorcontrib>Yang, Yuwang</creatorcontrib><creatorcontrib>Liao, Zhenqiang</creatorcontrib><title>A Data-Driven Packet Routing Algorithm for an Unmanned Aerial Vehicle Swarm: A Multi-Agent Reinforcement Learning Approach</title><title>IEEE wireless communications letters</title><addtitle>LWC</addtitle><description>Routing decisions made by unmanned aerial vehicle (UAV) swarms are affected by complex and dynamically changing topologies. A centralized routing algorithm imposes the entire computational burden on one module, and the high data dimensionality renders computation burdensome. In this letter, we develop a multi-agent reinforcement learning-based routing algorithm for a UAV swarm. The UAVs are trained in a data-driven manner to make distributed routing decisions. Factors that include channel quality, UAV movement, UAV overhead, and the extent of neighbor variation are incorporated into link quality assessment. Long short-term memory is used to improve the Actor and Critic networks, and more information on temporal continuity is added to facilitate adaptation to the dynamically changing environment. Simulations show that the proposed routing algorithm reduces data transmission delay and enhances the transmission rate compared with traditional routing algorithms.</description><subject>Algorithms</subject><subject>Artificial intelligence</subject><subject>Autonomous aerial vehicles</subject><subject>Changing environments</subject><subject>Data transmission</subject><subject>data-driven routing</subject><subject>Decisions</subject><subject>Heuristic algorithms</subject><subject>link quality assessment</subject><subject>LSTM</subject><subject>Machine learning</subject><subject>multi-agent reinforcement learning</subject><subject>Multiagent systems</subject><subject>Quality assessment</subject><subject>Route planning</subject><subject>Routing</subject><subject>Signal to noise ratio</subject><subject>Topology</subject><subject>Transmission rate (communications)</subject><subject>UAV swarm network</subject><subject>Unmanned aerial vehicles</subject><subject>Vehicle dynamics</subject><issn>2162-2337</issn><issn>2162-2345</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNo9kM9LwzAUx4soOHR3wUvAc2eSNm3jrWz-goqiTo8hTV-2zDadaavoX2_mxt7lvQffH_AJgjOCJ4Rgflm8TycUUzqJCGc8iQ6CESUJDWkUs8P9HaXHwbjrVthPggkl2Sj4zdFM9jKcOfMFFj1J9QE9em6H3tgFyutF60y_bJBuHZIWzW0jrYUK5eCMrNEbLI2qAb18S9dcoRw9DHVvwnwB1qeAsd6noNl8BUhn_0PXa9dKtTwNjrSsOxjv9kkwv7l-nd6FxePt_TQvQkU56cMKY1CUcAxaYo0rzUmpoiwBjWWaMZBlylmpVZJQrasMmAJcsaykMS65VCw6CS62ub72c4CuF6t2cNZXCppSyuIkTalX4a1KubbrHGixdqaR7kcQLDaQhYcsNpDFDrK3nG8tBgD2cp7FnBAW_QHln3jN</recordid><startdate>20221001</startdate><enddate>20221001</enddate><creator>Qiu, Xiulin</creator><creator>Xu, Lei</creator><creator>Wang, Ping</creator><creator>Yang, Yuwang</creator><creator>Liao, Zhenqiang</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SP</scope><scope>8FD</scope><scope>L7M</scope><orcidid>https://orcid.org/0000-0002-6636-8831</orcidid><orcidid>https://orcid.org/0000-0002-9306-5844</orcidid><orcidid>https://orcid.org/0000-0002-8535-6238</orcidid><orcidid>https://orcid.org/0000-0002-1599-5480</orcidid></search><sort><creationdate>20221001</creationdate><title>A Data-Driven Packet Routing Algorithm for an Unmanned Aerial Vehicle Swarm: A Multi-Agent Reinforcement Learning Approach</title><author>Qiu, Xiulin ; Xu, Lei ; Wang, Ping ; Yang, Yuwang ; Liao, Zhenqiang</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c291t-d00ec2190efa0f0df91bc386ef0a785eab795bfc662ffd8e5ce0d58b240b9ac53</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Algorithms</topic><topic>Artificial intelligence</topic><topic>Autonomous aerial vehicles</topic><topic>Changing environments</topic><topic>Data transmission</topic><topic>data-driven routing</topic><topic>Decisions</topic><topic>Heuristic algorithms</topic><topic>link quality assessment</topic><topic>LSTM</topic><topic>Machine learning</topic><topic>multi-agent reinforcement learning</topic><topic>Multiagent systems</topic><topic>Quality assessment</topic><topic>Route planning</topic><topic>Routing</topic><topic>Signal to noise ratio</topic><topic>Topology</topic><topic>Transmission rate (communications)</topic><topic>UAV swarm network</topic><topic>Unmanned aerial vehicles</topic><topic>Vehicle dynamics</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Qiu, Xiulin</creatorcontrib><creatorcontrib>Xu, Lei</creatorcontrib><creatorcontrib>Wang, Ping</creatorcontrib><creatorcontrib>Yang, Yuwang</creatorcontrib><creatorcontrib>Liao, Zhenqiang</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>Advanced Technologies Database with Aerospace</collection><jtitle>IEEE wireless communications letters</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Qiu, Xiulin</au><au>Xu, Lei</au><au>Wang, Ping</au><au>Yang, Yuwang</au><au>Liao, Zhenqiang</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A Data-Driven Packet Routing Algorithm for an Unmanned Aerial Vehicle Swarm: A Multi-Agent Reinforcement Learning Approach</atitle><jtitle>IEEE wireless communications letters</jtitle><stitle>LWC</stitle><date>2022-10-01</date><risdate>2022</risdate><volume>11</volume><issue>10</issue><spage>2160</spage><epage>2164</epage><pages>2160-2164</pages><issn>2162-2337</issn><eissn>2162-2345</eissn><coden>IWCLAF</coden><abstract>Routing decisions made by unmanned aerial vehicle (UAV) swarms are affected by complex and dynamically changing topologies. A centralized routing algorithm imposes the entire computational burden on one module, and the high data dimensionality renders computation burdensome. In this letter, we develop a multi-agent reinforcement learning-based routing algorithm for a UAV swarm. The UAVs are trained in a data-driven manner to make distributed routing decisions. Factors that include channel quality, UAV movement, UAV overhead, and the extent of neighbor variation are incorporated into link quality assessment. Long short-term memory is used to improve the Actor and Critic networks, and more information on temporal continuity is added to facilitate adaptation to the dynamically changing environment. Simulations show that the proposed routing algorithm reduces data transmission delay and enhances the transmission rate compared with traditional routing algorithms.</abstract><cop>Piscataway</cop><pub>IEEE</pub><doi>10.1109/LWC.2022.3195963</doi><tpages>5</tpages><orcidid>https://orcid.org/0000-0002-6636-8831</orcidid><orcidid>https://orcid.org/0000-0002-9306-5844</orcidid><orcidid>https://orcid.org/0000-0002-8535-6238</orcidid><orcidid>https://orcid.org/0000-0002-1599-5480</orcidid></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | ISSN: 2162-2337 |
| ispartof | IEEE wireless communications letters, 2022-10, Vol.11 (10), p.2160-2164 |
| issn | 2162-2337 2162-2345 |
| language | eng |
| recordid | cdi_proquest_journals_2722546772 |
| source | IEEE Electronic Library (IEL) |
| subjects | Algorithms Artificial intelligence Autonomous aerial vehicles Changing environments Data transmission data-driven routing Decisions Heuristic algorithms link quality assessment LSTM Machine learning multi-agent reinforcement learning Multiagent systems Quality assessment Route planning Routing Signal to noise ratio Topology Transmission rate (communications) UAV swarm network Unmanned aerial vehicles Vehicle dynamics |
| title | A Data-Driven Packet Routing Algorithm for an Unmanned Aerial Vehicle Swarm: A Multi-Agent Reinforcement Learning Approach |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-02T12%3A33%3A40IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20Data-Driven%20Packet%20Routing%20Algorithm%20for%20an%20Unmanned%20Aerial%20Vehicle%20Swarm:%20A%20Multi-Agent%20Reinforcement%20Learning%20Approach&rft.jtitle=IEEE%20wireless%20communications%20letters&rft.au=Qiu,%20Xiulin&rft.date=2022-10-01&rft.volume=11&rft.issue=10&rft.spage=2160&rft.epage=2164&rft.pages=2160-2164&rft.issn=2162-2337&rft.eissn=2162-2345&rft.coden=IWCLAF&rft_id=info:doi/10.1109/LWC.2022.3195963&rft_dat=%3Cproquest_RIE%3E2722546772%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2722546772&rft_id=info:pmid/&rft_ieee_id=9849115&rfr_iscdi=true |