Practical Resolution Methods for MDPs in Robotics Exemplified With Disassembly Planning
In this letter, we focus on finding practical resolution methods for Markov decision processes (MDPs) in robotics. Some of the main difficulties of applying MDPs to real-world robotics problems are: first, having to deal with huge state spaces; and second, designing a method that is robust enough to...
Gespeichert in:
Veröffentlicht in: | IEEE robotics and automation letters 2019-07, Vol.4 (3), p.2282-2288 |
---|---|
Hauptverfasser: | , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 2288 |
---|---|
container_issue | 3 |
container_start_page | 2282 |
container_title | IEEE robotics and automation letters |
container_volume | 4 |
creator | Suarez-Hernandez, Alejandro Torras, Carme Alenya, Guillem |
description | In this letter, we focus on finding practical resolution methods for Markov decision processes (MDPs) in robotics. Some of the main difficulties of applying MDPs to real-world robotics problems are: first, having to deal with huge state spaces; and second, designing a method that is robust enough to dead ends. These complications restrict or make more difficult the application of methods, such as value iteration, policy iteration, or labeled real-time dynamic programming (LRTDP). We see in determinization and heuristic search a way to successfully work around these problems. In addition, we believe that many practical use cases offer the opportunity to identify hierarchies of subtasks and solve smaller, simplified problems. We propose a decision-making unit that operates in a probabilistic planning setting through stochastic shortest path problems, which generalize the most common types of MDPs. Our decision-making unit combines: first, automatic hierarchical organization of subtasks; and second, on-line resolution via determinization. We argue that several applications of planning benefit from these two strategies. We exemplify our approach with a robotized disassembly application. The disassembly problem is modeled in probabilistic planning definition language, and serves to define our experiments. Our results show many advantages of our method over LRTDP, such as a better capability to handle problems with large state spaces and state definitions that change when new fluents are discovered. |
doi_str_mv | 10.1109/LRA.2019.2901905 |
format | Article |
fullrecord | <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_ieee_primary_8653968</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>8653968</ieee_id><sourcerecordid>2298321828</sourcerecordid><originalsourceid>FETCH-LOGICAL-c333t-2c4d1f7ae00fe294c2893827af1e61cfb2af9e485a764933cfa692085f6ed26d3</originalsourceid><addsrcrecordid>eNpNkE1rAjEQhkNpoWK9F3oJ9Lw2H7vZ5ChqP0CpSIvHELOTGlk3Nlmh_vuuKKWXeefwvDPwIHRPyZBSop5my9GQEaqGTHWTFFeox3hZZrwU4vrffosGKW0JIbRgJVdFD60W0djWW1PjJaRQH1ofGjyHdhOqhF2IeD5ZJOwbvAzr0IEJT39gt6-981DhlW83eOKTSQl26_qIF7VpGt983aEbZ-oEg0v20efz9GP8ms3eX97Go1lmOedtxmxeUVcaIMQBU7llUnHJSuMoCGrdmhmnIJeFKUWuOLfOCMWILJyAiomK99Hj-e4-hu8DpFZvwyE23UvNmJKcUclkR5EzZWNIKYLT--h3Jh41JfpkUHcG9cmgvhjsKg_nigeAP1yKgish-S-3bWxB</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2298321828</pqid></control><display><type>article</type><title>Practical Resolution Methods for MDPs in Robotics Exemplified With Disassembly Planning</title><source>IEEE Electronic Library (IEL)</source><creator>Suarez-Hernandez, Alejandro ; Torras, Carme ; Alenya, Guillem</creator><creatorcontrib>Suarez-Hernandez, Alejandro ; Torras, Carme ; Alenya, Guillem</creatorcontrib><description>In this letter, we focus on finding practical resolution methods for Markov decision processes (MDPs) in robotics. Some of the main difficulties of applying MDPs to real-world robotics problems are: first, having to deal with huge state spaces; and second, designing a method that is robust enough to dead ends. These complications restrict or make more difficult the application of methods, such as value iteration, policy iteration, or labeled real-time dynamic programming (LRTDP). We see in determinization and heuristic search a way to successfully work around these problems. In addition, we believe that many practical use cases offer the opportunity to identify hierarchies of subtasks and solve smaller, simplified problems. We propose a decision-making unit that operates in a probabilistic planning setting through stochastic shortest path problems, which generalize the most common types of MDPs. Our decision-making unit combines: first, automatic hierarchical organization of subtasks; and second, on-line resolution via determinization. We argue that several applications of planning benefit from these two strategies. We exemplify our approach with a robotized disassembly application. The disassembly problem is modeled in probabilistic planning definition language, and serves to define our experiments. Our results show many advantages of our method over LRTDP, such as a better capability to handle problems with large state spaces and state definitions that change when new fluents are discovered.</description><identifier>ISSN: 2377-3766</identifier><identifier>EISSN: 2377-3766</identifier><identifier>DOI: 10.1109/LRA.2019.2901905</identifier><identifier>CODEN: IRALC6</identifier><language>eng</language><publisher>Piscataway: IEEE</publisher><subject>Decision making ; Dismantling ; Dynamic programming ; Fasteners ; Hierarchies ; hybrid logical/dynamical planning and verification ; Iterative methods ; Markov processes ; Planning ; Probabilistic logic ; Probability theory ; Real-time programming ; Robot kinematics ; Robotics ; scheduling and coordination ; Shortest path planning ; Task analysis ; task planning</subject><ispartof>IEEE robotics and automation letters, 2019-07, Vol.4 (3), p.2282-2288</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2019</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c333t-2c4d1f7ae00fe294c2893827af1e61cfb2af9e485a764933cfa692085f6ed26d3</citedby><cites>FETCH-LOGICAL-c333t-2c4d1f7ae00fe294c2893827af1e61cfb2af9e485a764933cfa692085f6ed26d3</cites><orcidid>0000-0002-6018-154X ; 0000-0003-1611-614X ; 0000-0002-2933-398X</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/8653968$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>315,781,785,797,27929,27930,54763</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/8653968$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Suarez-Hernandez, Alejandro</creatorcontrib><creatorcontrib>Torras, Carme</creatorcontrib><creatorcontrib>Alenya, Guillem</creatorcontrib><title>Practical Resolution Methods for MDPs in Robotics Exemplified With Disassembly Planning</title><title>IEEE robotics and automation letters</title><addtitle>LRA</addtitle><description>In this letter, we focus on finding practical resolution methods for Markov decision processes (MDPs) in robotics. Some of the main difficulties of applying MDPs to real-world robotics problems are: first, having to deal with huge state spaces; and second, designing a method that is robust enough to dead ends. These complications restrict or make more difficult the application of methods, such as value iteration, policy iteration, or labeled real-time dynamic programming (LRTDP). We see in determinization and heuristic search a way to successfully work around these problems. In addition, we believe that many practical use cases offer the opportunity to identify hierarchies of subtasks and solve smaller, simplified problems. We propose a decision-making unit that operates in a probabilistic planning setting through stochastic shortest path problems, which generalize the most common types of MDPs. Our decision-making unit combines: first, automatic hierarchical organization of subtasks; and second, on-line resolution via determinization. We argue that several applications of planning benefit from these two strategies. We exemplify our approach with a robotized disassembly application. The disassembly problem is modeled in probabilistic planning definition language, and serves to define our experiments. Our results show many advantages of our method over LRTDP, such as a better capability to handle problems with large state spaces and state definitions that change when new fluents are discovered.</description><subject>Decision making</subject><subject>Dismantling</subject><subject>Dynamic programming</subject><subject>Fasteners</subject><subject>Hierarchies</subject><subject>hybrid logical/dynamical planning and verification</subject><subject>Iterative methods</subject><subject>Markov processes</subject><subject>Planning</subject><subject>Probabilistic logic</subject><subject>Probability theory</subject><subject>Real-time programming</subject><subject>Robot kinematics</subject><subject>Robotics</subject><subject>scheduling and coordination</subject><subject>Shortest path planning</subject><subject>Task analysis</subject><subject>task planning</subject><issn>2377-3766</issn><issn>2377-3766</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNpNkE1rAjEQhkNpoWK9F3oJ9Lw2H7vZ5ChqP0CpSIvHELOTGlk3Nlmh_vuuKKWXeefwvDPwIHRPyZBSop5my9GQEaqGTHWTFFeox3hZZrwU4vrffosGKW0JIbRgJVdFD60W0djWW1PjJaRQH1ofGjyHdhOqhF2IeD5ZJOwbvAzr0IEJT39gt6-981DhlW83eOKTSQl26_qIF7VpGt983aEbZ-oEg0v20efz9GP8ms3eX97Go1lmOedtxmxeUVcaIMQBU7llUnHJSuMoCGrdmhmnIJeFKUWuOLfOCMWILJyAiomK99Hj-e4-hu8DpFZvwyE23UvNmJKcUclkR5EzZWNIKYLT--h3Jh41JfpkUHcG9cmgvhjsKg_nigeAP1yKgish-S-3bWxB</recordid><startdate>20190701</startdate><enddate>20190701</enddate><creator>Suarez-Hernandez, Alejandro</creator><creator>Torras, Carme</creator><creator>Alenya, Guillem</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0002-6018-154X</orcidid><orcidid>https://orcid.org/0000-0003-1611-614X</orcidid><orcidid>https://orcid.org/0000-0002-2933-398X</orcidid></search><sort><creationdate>20190701</creationdate><title>Practical Resolution Methods for MDPs in Robotics Exemplified With Disassembly Planning</title><author>Suarez-Hernandez, Alejandro ; Torras, Carme ; Alenya, Guillem</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c333t-2c4d1f7ae00fe294c2893827af1e61cfb2af9e485a764933cfa692085f6ed26d3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Decision making</topic><topic>Dismantling</topic><topic>Dynamic programming</topic><topic>Fasteners</topic><topic>Hierarchies</topic><topic>hybrid logical/dynamical planning and verification</topic><topic>Iterative methods</topic><topic>Markov processes</topic><topic>Planning</topic><topic>Probabilistic logic</topic><topic>Probability theory</topic><topic>Real-time programming</topic><topic>Robot kinematics</topic><topic>Robotics</topic><topic>scheduling and coordination</topic><topic>Shortest path planning</topic><topic>Task analysis</topic><topic>task planning</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Suarez-Hernandez, Alejandro</creatorcontrib><creatorcontrib>Torras, Carme</creatorcontrib><creatorcontrib>Alenya, Guillem</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE robotics and automation letters</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Suarez-Hernandez, Alejandro</au><au>Torras, Carme</au><au>Alenya, Guillem</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Practical Resolution Methods for MDPs in Robotics Exemplified With Disassembly Planning</atitle><jtitle>IEEE robotics and automation letters</jtitle><stitle>LRA</stitle><date>2019-07-01</date><risdate>2019</risdate><volume>4</volume><issue>3</issue><spage>2282</spage><epage>2288</epage><pages>2282-2288</pages><issn>2377-3766</issn><eissn>2377-3766</eissn><coden>IRALC6</coden><abstract>In this letter, we focus on finding practical resolution methods for Markov decision processes (MDPs) in robotics. Some of the main difficulties of applying MDPs to real-world robotics problems are: first, having to deal with huge state spaces; and second, designing a method that is robust enough to dead ends. These complications restrict or make more difficult the application of methods, such as value iteration, policy iteration, or labeled real-time dynamic programming (LRTDP). We see in determinization and heuristic search a way to successfully work around these problems. In addition, we believe that many practical use cases offer the opportunity to identify hierarchies of subtasks and solve smaller, simplified problems. We propose a decision-making unit that operates in a probabilistic planning setting through stochastic shortest path problems, which generalize the most common types of MDPs. Our decision-making unit combines: first, automatic hierarchical organization of subtasks; and second, on-line resolution via determinization. We argue that several applications of planning benefit from these two strategies. We exemplify our approach with a robotized disassembly application. The disassembly problem is modeled in probabilistic planning definition language, and serves to define our experiments. Our results show many advantages of our method over LRTDP, such as a better capability to handle problems with large state spaces and state definitions that change when new fluents are discovered.</abstract><cop>Piscataway</cop><pub>IEEE</pub><doi>10.1109/LRA.2019.2901905</doi><tpages>7</tpages><orcidid>https://orcid.org/0000-0002-6018-154X</orcidid><orcidid>https://orcid.org/0000-0003-1611-614X</orcidid><orcidid>https://orcid.org/0000-0002-2933-398X</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext_linktorsrc |
identifier | ISSN: 2377-3766 |
ispartof | IEEE robotics and automation letters, 2019-07, Vol.4 (3), p.2282-2288 |
issn | 2377-3766 2377-3766 |
language | eng |
recordid | cdi_ieee_primary_8653968 |
source | IEEE Electronic Library (IEL) |
subjects | Decision making Dismantling Dynamic programming Fasteners Hierarchies hybrid logical/dynamical planning and verification Iterative methods Markov processes Planning Probabilistic logic Probability theory Real-time programming Robot kinematics Robotics scheduling and coordination Shortest path planning Task analysis task planning |
title | Practical Resolution Methods for MDPs in Robotics Exemplified With Disassembly Planning |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-15T15%3A46%3A38IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Practical%20Resolution%20Methods%20for%20MDPs%20in%20Robotics%20Exemplified%20With%20Disassembly%20Planning&rft.jtitle=IEEE%20robotics%20and%20automation%20letters&rft.au=Suarez-Hernandez,%20Alejandro&rft.date=2019-07-01&rft.volume=4&rft.issue=3&rft.spage=2282&rft.epage=2288&rft.pages=2282-2288&rft.issn=2377-3766&rft.eissn=2377-3766&rft.coden=IRALC6&rft_id=info:doi/10.1109/LRA.2019.2901905&rft_dat=%3Cproquest_RIE%3E2298321828%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2298321828&rft_id=info:pmid/&rft_ieee_id=8653968&rfr_iscdi=true |