Hierarchical Planning and Control for Box Loco-Manipulation
Humans perform everyday tasks using a combination of locomotion and manipulation skills. Building a system that can handle both skills is essential to creating virtual humans. We present a physically-simulated human capable of solving box rearrangement tasks, which requires a combination of both ski...
Gespeichert in:
| Veröffentlicht in: | Proceedings of the ACM on computer graphics and interactive techniques 2023-08, Vol.6 (3), p.1-18, Article 31 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 18 |
|---|---|
| container_issue | 3 |
| container_start_page | 1 |
| container_title | Proceedings of the ACM on computer graphics and interactive techniques |
| container_volume | 6 |
| creator | Xie, Zhaoming Tseng, Jonathan Starke, Sebastian van de Panne, Michiel Liu, C. Karen |
| description | Humans perform everyday tasks using a combination of locomotion and manipulation skills. Building a system that can handle both skills is essential to creating virtual humans. We present a physically-simulated human capable of solving box rearrangement tasks, which requires a combination of both skills. We propose a hierarchical control architecture, where each level solves the task at a different level of abstraction, and the result is a physics-based simulated virtual human capable of rearranging boxes in a cluttered environment. The control architecture integrates a planner, diffusion models, and physics-based motion imitation of sparse motion clips using deep reinforcement learning. Boxes can vary in size, weight, shape, and placement height. Code and trained control policies are provided. |
| doi_str_mv | 10.1145/3606931 |
| format | Article |
| fullrecord | <record><control><sourceid>acm_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1145_3606931</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3606931</sourcerecordid><originalsourceid>FETCH-LOGICAL-a206t-8d861250059ed19eee6d93a3879871ecb2fcf1f4075cff3782191de8aa43bfe63</originalsourceid><addsrcrecordid>eNpNj01LxDAURYMoOIyDe1fZuarmJW0-cKVFHaGiC12XN2mikU4yJBX036vMKK7uhXu4cAg5BnYGUDfnQjJpBOyRGW-UqiQYsf-vH5JFKW-MMQ61kVzMyMUyuIzZvgaLI30cMcYQXyjGgbYpTjmN1KdMr9IH7ZJN1T3GsHkfcQopHpEDj2Nxi13OyfPN9VO7rLqH27v2squQMzlVetASeMNYY9wAxjknByNQaGW0AmdX3FsPvmaqsd4LpTkYGJxGrMXKOynm5HT7a3MqJTvfb3JYY_7sgfU_2v1O-5s82ZJo13_Q7_gFgEtRDw</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Hierarchical Planning and Control for Box Loco-Manipulation</title><source>Association for Computing Machinery</source><creator>Xie, Zhaoming ; Tseng, Jonathan ; Starke, Sebastian ; van de Panne, Michiel ; Liu, C. Karen</creator><creatorcontrib>Xie, Zhaoming ; Tseng, Jonathan ; Starke, Sebastian ; van de Panne, Michiel ; Liu, C. Karen</creatorcontrib><description>Humans perform everyday tasks using a combination of locomotion and manipulation skills. Building a system that can handle both skills is essential to creating virtual humans. We present a physically-simulated human capable of solving box rearrangement tasks, which requires a combination of both skills. We propose a hierarchical control architecture, where each level solves the task at a different level of abstraction, and the result is a physics-based simulated virtual human capable of rearranging boxes in a cluttered environment. The control architecture integrates a planner, diffusion models, and physics-based motion imitation of sparse motion clips using deep reinforcement learning. Boxes can vary in size, weight, shape, and placement height. Code and trained control policies are provided.</description><identifier>ISSN: 2577-6193</identifier><identifier>EISSN: 2577-6193</identifier><identifier>DOI: 10.1145/3606931</identifier><language>eng</language><publisher>New York, NY, USA: ACM</publisher><subject>Animation ; Computer graphics ; Computing methodologies ; Learning paradigms ; Machine learning ; Physical simulation ; Reinforcement learning</subject><ispartof>Proceedings of the ACM on computer graphics and interactive techniques, 2023-08, Vol.6 (3), p.1-18, Article 31</ispartof><rights>ACM</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-a206t-8d861250059ed19eee6d93a3879871ecb2fcf1f4075cff3782191de8aa43bfe63</cites><orcidid>0000-0002-9123-3672 ; 0000-0001-5926-0905 ; 0000-0002-4519-4326 ; 0000-0003-1948-1767 ; 0000-0001-8535-8324</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://dl.acm.org/doi/pdf/10.1145/3606931$$EPDF$$P50$$Gacm$$H</linktopdf><link.rule.ids>314,780,784,2282,27924,27925,40196,76100</link.rule.ids></links><search><creatorcontrib>Xie, Zhaoming</creatorcontrib><creatorcontrib>Tseng, Jonathan</creatorcontrib><creatorcontrib>Starke, Sebastian</creatorcontrib><creatorcontrib>van de Panne, Michiel</creatorcontrib><creatorcontrib>Liu, C. Karen</creatorcontrib><title>Hierarchical Planning and Control for Box Loco-Manipulation</title><title>Proceedings of the ACM on computer graphics and interactive techniques</title><addtitle>ACM PACMCGIT</addtitle><description>Humans perform everyday tasks using a combination of locomotion and manipulation skills. Building a system that can handle both skills is essential to creating virtual humans. We present a physically-simulated human capable of solving box rearrangement tasks, which requires a combination of both skills. We propose a hierarchical control architecture, where each level solves the task at a different level of abstraction, and the result is a physics-based simulated virtual human capable of rearranging boxes in a cluttered environment. The control architecture integrates a planner, diffusion models, and physics-based motion imitation of sparse motion clips using deep reinforcement learning. Boxes can vary in size, weight, shape, and placement height. Code and trained control policies are provided.</description><subject>Animation</subject><subject>Computer graphics</subject><subject>Computing methodologies</subject><subject>Learning paradigms</subject><subject>Machine learning</subject><subject>Physical simulation</subject><subject>Reinforcement learning</subject><issn>2577-6193</issn><issn>2577-6193</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><recordid>eNpNj01LxDAURYMoOIyDe1fZuarmJW0-cKVFHaGiC12XN2mikU4yJBX036vMKK7uhXu4cAg5BnYGUDfnQjJpBOyRGW-UqiQYsf-vH5JFKW-MMQ61kVzMyMUyuIzZvgaLI30cMcYQXyjGgbYpTjmN1KdMr9IH7ZJN1T3GsHkfcQopHpEDj2Nxi13OyfPN9VO7rLqH27v2squQMzlVetASeMNYY9wAxjknByNQaGW0AmdX3FsPvmaqsd4LpTkYGJxGrMXKOynm5HT7a3MqJTvfb3JYY_7sgfU_2v1O-5s82ZJo13_Q7_gFgEtRDw</recordid><startdate>20230823</startdate><enddate>20230823</enddate><creator>Xie, Zhaoming</creator><creator>Tseng, Jonathan</creator><creator>Starke, Sebastian</creator><creator>van de Panne, Michiel</creator><creator>Liu, C. Karen</creator><general>ACM</general><scope>AAYXX</scope><scope>CITATION</scope><orcidid>https://orcid.org/0000-0002-9123-3672</orcidid><orcidid>https://orcid.org/0000-0001-5926-0905</orcidid><orcidid>https://orcid.org/0000-0002-4519-4326</orcidid><orcidid>https://orcid.org/0000-0003-1948-1767</orcidid><orcidid>https://orcid.org/0000-0001-8535-8324</orcidid></search><sort><creationdate>20230823</creationdate><title>Hierarchical Planning and Control for Box Loco-Manipulation</title><author>Xie, Zhaoming ; Tseng, Jonathan ; Starke, Sebastian ; van de Panne, Michiel ; Liu, C. Karen</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a206t-8d861250059ed19eee6d93a3879871ecb2fcf1f4075cff3782191de8aa43bfe63</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Animation</topic><topic>Computer graphics</topic><topic>Computing methodologies</topic><topic>Learning paradigms</topic><topic>Machine learning</topic><topic>Physical simulation</topic><topic>Reinforcement learning</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Xie, Zhaoming</creatorcontrib><creatorcontrib>Tseng, Jonathan</creatorcontrib><creatorcontrib>Starke, Sebastian</creatorcontrib><creatorcontrib>van de Panne, Michiel</creatorcontrib><creatorcontrib>Liu, C. Karen</creatorcontrib><collection>CrossRef</collection><jtitle>Proceedings of the ACM on computer graphics and interactive techniques</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Xie, Zhaoming</au><au>Tseng, Jonathan</au><au>Starke, Sebastian</au><au>van de Panne, Michiel</au><au>Liu, C. Karen</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Hierarchical Planning and Control for Box Loco-Manipulation</atitle><jtitle>Proceedings of the ACM on computer graphics and interactive techniques</jtitle><stitle>ACM PACMCGIT</stitle><date>2023-08-23</date><risdate>2023</risdate><volume>6</volume><issue>3</issue><spage>1</spage><epage>18</epage><pages>1-18</pages><artnum>31</artnum><issn>2577-6193</issn><eissn>2577-6193</eissn><abstract>Humans perform everyday tasks using a combination of locomotion and manipulation skills. Building a system that can handle both skills is essential to creating virtual humans. We present a physically-simulated human capable of solving box rearrangement tasks, which requires a combination of both skills. We propose a hierarchical control architecture, where each level solves the task at a different level of abstraction, and the result is a physics-based simulated virtual human capable of rearranging boxes in a cluttered environment. The control architecture integrates a planner, diffusion models, and physics-based motion imitation of sparse motion clips using deep reinforcement learning. Boxes can vary in size, weight, shape, and placement height. Code and trained control policies are provided.</abstract><cop>New York, NY, USA</cop><pub>ACM</pub><doi>10.1145/3606931</doi><tpages>18</tpages><orcidid>https://orcid.org/0000-0002-9123-3672</orcidid><orcidid>https://orcid.org/0000-0001-5926-0905</orcidid><orcidid>https://orcid.org/0000-0002-4519-4326</orcidid><orcidid>https://orcid.org/0000-0003-1948-1767</orcidid><orcidid>https://orcid.org/0000-0001-8535-8324</orcidid></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 2577-6193 |
| ispartof | Proceedings of the ACM on computer graphics and interactive techniques, 2023-08, Vol.6 (3), p.1-18, Article 31 |
| issn | 2577-6193 2577-6193 |
| language | eng |
| recordid | cdi_crossref_primary_10_1145_3606931 |
| source | Association for Computing Machinery |
| subjects | Animation Computer graphics Computing methodologies Learning paradigms Machine learning Physical simulation Reinforcement learning |
| title | Hierarchical Planning and Control for Box Loco-Manipulation |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-07T17%3A47%3A22IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-acm_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Hierarchical%20Planning%20and%20Control%20for%20Box%20Loco-Manipulation&rft.jtitle=Proceedings%20of%20the%20ACM%20on%20computer%20graphics%20and%20interactive%20techniques&rft.au=Xie,%20Zhaoming&rft.date=2023-08-23&rft.volume=6&rft.issue=3&rft.spage=1&rft.epage=18&rft.pages=1-18&rft.artnum=31&rft.issn=2577-6193&rft.eissn=2577-6193&rft_id=info:doi/10.1145/3606931&rft_dat=%3Cacm_cross%3E3606931%3C/acm_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |