Multiagent, Multitarget Path Planning in Markov Decision Processes

Missions for autonomous systems often require agents to visit multiple targets in complex operating conditions. This work considers the problem of visiting a set of targets in minimum time by a team of noncommunicating agents in a Markov decision process (MDP). The single-agent problem is at least N...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on automatic control 2023-12, Vol.68 (12), p.7560-7574
Hauptverfasser: Nawaz, Farhad, Ornik, Melkior
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 7574
container_issue 12
container_start_page 7560
container_title IEEE transactions on automatic control
container_volume 68
creator Nawaz, Farhad
Ornik, Melkior
description Missions for autonomous systems often require agents to visit multiple targets in complex operating conditions. This work considers the problem of visiting a set of targets in minimum time by a team of noncommunicating agents in a Markov decision process (MDP). The single-agent problem is at least NP-complete by reducing it to a Hamiltonian path problem. We first discuss an optimal algorithm based on Bellman's optimality equation that is exponential in the number of target states. Then, we tradeoff optimality for time complexity by presenting a suboptimal algorithm that is polynomial at each time step. We prove that the proposed algorithm generates optimal policies for certain classes of MDPs. Extending our procedure to the multiagent case, we propose a target partitioning algorithm that approximately minimizes the expected time to visit the targets. We prove that our algorithm generates optimal partitions for clustered target scenarios. We present the performance of our algorithms on random MDPs and gridworld environments inspired by ocean dynamics. We show that our algorithms are much faster than the optimal procedure and more optimal than the currently available heuristic.
doi_str_mv 10.1109/TAC.2023.3286807
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2899221465</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2899221465</sourcerecordid><originalsourceid>FETCH-LOGICAL-c224t-f6a5da523dc1cce2eaa6450005e9db82bda73a2fd77933ee342030bd3966e6203</originalsourceid><addsrcrecordid>eNotkM1PAjEQxRujiYjePTbx6mI73Xa3R8TPBCIHPDelO4uL2MW2mPjfW4TTvMm8vDf5EXLN2Yhzpu8W48kIGIiRgFrVrDohAy5lXYAEcUoGjPG60Pl0Ti5iXOdVlSUfkPvZbpM6u0Kfbum_TjasMNG5TR90vrHed35FO09nNnz2P_QBXRe73tN56B3GiPGSnLV2E_HqOIfk_elxMXkppm_Pr5PxtHAAZSpaZWVj8zeN484hoLWqlIwxibpZ1rBsbCUstE1VaSEQRQlMsGUjtFKosh6Sm0PuNvTfO4zJrPtd8LnSQK01AC-VzC52cLnQxxiwNdvQfdnwazgze1ImkzJ7UuZISvwBi5JbBw</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2899221465</pqid></control><display><type>article</type><title>Multiagent, Multitarget Path Planning in Markov Decision Processes</title><source>IEEE Electronic Library (IEL)</source><creator>Nawaz, Farhad ; Ornik, Melkior</creator><creatorcontrib>Nawaz, Farhad ; Ornik, Melkior</creatorcontrib><description>Missions for autonomous systems often require agents to visit multiple targets in complex operating conditions. This work considers the problem of visiting a set of targets in minimum time by a team of noncommunicating agents in a Markov decision process (MDP). The single-agent problem is at least NP-complete by reducing it to a Hamiltonian path problem. We first discuss an optimal algorithm based on Bellman's optimality equation that is exponential in the number of target states. Then, we tradeoff optimality for time complexity by presenting a suboptimal algorithm that is polynomial at each time step. We prove that the proposed algorithm generates optimal policies for certain classes of MDPs. Extending our procedure to the multiagent case, we propose a target partitioning algorithm that approximately minimizes the expected time to visit the targets. We prove that our algorithm generates optimal partitions for clustered target scenarios. We present the performance of our algorithms on random MDPs and gridworld environments inspired by ocean dynamics. We show that our algorithms are much faster than the optimal procedure and more optimal than the currently available heuristic.</description><identifier>ISSN: 0018-9286</identifier><identifier>EISSN: 1558-2523</identifier><identifier>DOI: 10.1109/TAC.2023.3286807</identifier><language>eng</language><publisher>New York: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</publisher><subject>Algorithms ; Complexity ; Markov processes ; Multiagent systems ; Ocean dynamics ; Optimization ; Path planning ; Polynomials</subject><ispartof>IEEE transactions on automatic control, 2023-12, Vol.68 (12), p.7560-7574</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c224t-f6a5da523dc1cce2eaa6450005e9db82bda73a2fd77933ee342030bd3966e6203</cites><orcidid>0000-0002-8510-8787 ; 0000-0001-5390-9363</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,27901,27902</link.rule.ids></links><search><creatorcontrib>Nawaz, Farhad</creatorcontrib><creatorcontrib>Ornik, Melkior</creatorcontrib><title>Multiagent, Multitarget Path Planning in Markov Decision Processes</title><title>IEEE transactions on automatic control</title><description>Missions for autonomous systems often require agents to visit multiple targets in complex operating conditions. This work considers the problem of visiting a set of targets in minimum time by a team of noncommunicating agents in a Markov decision process (MDP). The single-agent problem is at least NP-complete by reducing it to a Hamiltonian path problem. We first discuss an optimal algorithm based on Bellman's optimality equation that is exponential in the number of target states. Then, we tradeoff optimality for time complexity by presenting a suboptimal algorithm that is polynomial at each time step. We prove that the proposed algorithm generates optimal policies for certain classes of MDPs. Extending our procedure to the multiagent case, we propose a target partitioning algorithm that approximately minimizes the expected time to visit the targets. We prove that our algorithm generates optimal partitions for clustered target scenarios. We present the performance of our algorithms on random MDPs and gridworld environments inspired by ocean dynamics. We show that our algorithms are much faster than the optimal procedure and more optimal than the currently available heuristic.</description><subject>Algorithms</subject><subject>Complexity</subject><subject>Markov processes</subject><subject>Multiagent systems</subject><subject>Ocean dynamics</subject><subject>Optimization</subject><subject>Path planning</subject><subject>Polynomials</subject><issn>0018-9286</issn><issn>1558-2523</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><recordid>eNotkM1PAjEQxRujiYjePTbx6mI73Xa3R8TPBCIHPDelO4uL2MW2mPjfW4TTvMm8vDf5EXLN2Yhzpu8W48kIGIiRgFrVrDohAy5lXYAEcUoGjPG60Pl0Ti5iXOdVlSUfkPvZbpM6u0Kfbum_TjasMNG5TR90vrHed35FO09nNnz2P_QBXRe73tN56B3GiPGSnLV2E_HqOIfk_elxMXkppm_Pr5PxtHAAZSpaZWVj8zeN484hoLWqlIwxibpZ1rBsbCUstE1VaSEQRQlMsGUjtFKosh6Sm0PuNvTfO4zJrPtd8LnSQK01AC-VzC52cLnQxxiwNdvQfdnwazgze1ImkzJ7UuZISvwBi5JbBw</recordid><startdate>20231201</startdate><enddate>20231201</enddate><creator>Nawaz, Farhad</creator><creator>Ornik, Melkior</creator><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>7TB</scope><scope>8FD</scope><scope>FR3</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0002-8510-8787</orcidid><orcidid>https://orcid.org/0000-0001-5390-9363</orcidid></search><sort><creationdate>20231201</creationdate><title>Multiagent, Multitarget Path Planning in Markov Decision Processes</title><author>Nawaz, Farhad ; Ornik, Melkior</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c224t-f6a5da523dc1cce2eaa6450005e9db82bda73a2fd77933ee342030bd3966e6203</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Algorithms</topic><topic>Complexity</topic><topic>Markov processes</topic><topic>Multiagent systems</topic><topic>Ocean dynamics</topic><topic>Optimization</topic><topic>Path planning</topic><topic>Polynomials</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Nawaz, Farhad</creatorcontrib><creatorcontrib>Ornik, Melkior</creatorcontrib><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Mechanical &amp; Transportation Engineering Abstracts</collection><collection>Technology Research Database</collection><collection>Engineering Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE transactions on automatic control</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Nawaz, Farhad</au><au>Ornik, Melkior</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Multiagent, Multitarget Path Planning in Markov Decision Processes</atitle><jtitle>IEEE transactions on automatic control</jtitle><date>2023-12-01</date><risdate>2023</risdate><volume>68</volume><issue>12</issue><spage>7560</spage><epage>7574</epage><pages>7560-7574</pages><issn>0018-9286</issn><eissn>1558-2523</eissn><abstract>Missions for autonomous systems often require agents to visit multiple targets in complex operating conditions. This work considers the problem of visiting a set of targets in minimum time by a team of noncommunicating agents in a Markov decision process (MDP). The single-agent problem is at least NP-complete by reducing it to a Hamiltonian path problem. We first discuss an optimal algorithm based on Bellman's optimality equation that is exponential in the number of target states. Then, we tradeoff optimality for time complexity by presenting a suboptimal algorithm that is polynomial at each time step. We prove that the proposed algorithm generates optimal policies for certain classes of MDPs. Extending our procedure to the multiagent case, we propose a target partitioning algorithm that approximately minimizes the expected time to visit the targets. We prove that our algorithm generates optimal partitions for clustered target scenarios. We present the performance of our algorithms on random MDPs and gridworld environments inspired by ocean dynamics. We show that our algorithms are much faster than the optimal procedure and more optimal than the currently available heuristic.</abstract><cop>New York</cop><pub>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</pub><doi>10.1109/TAC.2023.3286807</doi><tpages>15</tpages><orcidid>https://orcid.org/0000-0002-8510-8787</orcidid><orcidid>https://orcid.org/0000-0001-5390-9363</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 0018-9286
ispartof IEEE transactions on automatic control, 2023-12, Vol.68 (12), p.7560-7574
issn 0018-9286
1558-2523
language eng
recordid cdi_proquest_journals_2899221465
source IEEE Electronic Library (IEL)
subjects Algorithms
Complexity
Markov processes
Multiagent systems
Ocean dynamics
Optimization
Path planning
Polynomials
title Multiagent, Multitarget Path Planning in Markov Decision Processes
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-20T04%3A54%3A37IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Multiagent,%20Multitarget%20Path%20Planning%20in%20Markov%20Decision%20Processes&rft.jtitle=IEEE%20transactions%20on%20automatic%20control&rft.au=Nawaz,%20Farhad&rft.date=2023-12-01&rft.volume=68&rft.issue=12&rft.spage=7560&rft.epage=7574&rft.pages=7560-7574&rft.issn=0018-9286&rft.eissn=1558-2523&rft_id=info:doi/10.1109/TAC.2023.3286807&rft_dat=%3Cproquest_cross%3E2899221465%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2899221465&rft_id=info:pmid/&rfr_iscdi=true