Evolution and learning in differentiable robots

The automatic design of robots has existed for 30 years but has been constricted by serial non-differentiable design evaluations, premature convergence to simple bodies or clumsy behaviors, and a lack of sim2real transfer to physical machines. Thus, here we employ massively-parallel differentiable s...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Strgar, Luke, Matthews, David, Hummer, Tyler, Kriegman, Sam
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Strgar, Luke
Matthews, David
Hummer, Tyler
Kriegman, Sam
description The automatic design of robots has existed for 30 years but has been constricted by serial non-differentiable design evaluations, premature convergence to simple bodies or clumsy behaviors, and a lack of sim2real transfer to physical machines. Thus, here we employ massively-parallel differentiable simulations to rapidly and simultaneously optimize individual neural control of behavior across a large population of candidate body plans and return a fitness score for each design based on the performance of its fully optimized behavior. Non-differentiable changes to the mechanical structure of each robot in the population -- mutations that rearrange, combine, add, or remove body parts -- were applied by a genetic algorithm in an outer loop of search, generating a continuous flow of novel morphologies with highly-coordinated and graceful behaviors honed by gradient descent. This enabled the exploration of several orders-of-magnitude more designs than all previous methods, despite the fact that robots here have the potential to be much more complex, in terms of number of independent motors, than those in prior studies. We found that evolution reliably produces ``increasingly differentiable'' robots: body plans that smooth the loss landscape in which learning operates and thereby provide better training paths toward performant behaviors. Finally, one of the highly differentiable morphologies discovered in simulation was realized as a physical robot and shown to retain its optimized behavior. This provides a cyberphysical platform to investigate the relationship between evolution and learning in biological systems and broadens our understanding of how a robot's physical structure can influence the ability to train policies for it. Videos and code at https://sites.google.com/view/eldir.
doi_str_mv 10.48550/arxiv.2405.14712
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2405_14712</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2405_14712</sourcerecordid><originalsourceid>FETCH-LOGICAL-a672-f9db02d7dc0edda326023348aa12cd83ca6d03dd42aaa5487497bdccf1c17cdc3</originalsourceid><addsrcrecordid>eNotzrsKwjAUgOEsDqI-gJN5gdbc2rSjlHoBwcW9nOYkEqippFX07cXL9G8_HyFLzlJVZBlbQ3z6RyoUy1KuNBdTsq4ffXcffR8oBKSdhRh8uFAfKHrnbLRh9NB2lsa-7cdhTiYOusEu_p2R87Y-V_vkeNodqs0xgVyLxJXYMoEaDbOIIEXOhJSqAODCYCEN5MgkohIAkKlCq1K3aIzjhmuDRs7I6rf9iptb9FeIr-Yjb75y-QZYgT7l</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Evolution and learning in differentiable robots</title><source>arXiv.org</source><creator>Strgar, Luke ; Matthews, David ; Hummer, Tyler ; Kriegman, Sam</creator><creatorcontrib>Strgar, Luke ; Matthews, David ; Hummer, Tyler ; Kriegman, Sam</creatorcontrib><description>The automatic design of robots has existed for 30 years but has been constricted by serial non-differentiable design evaluations, premature convergence to simple bodies or clumsy behaviors, and a lack of sim2real transfer to physical machines. Thus, here we employ massively-parallel differentiable simulations to rapidly and simultaneously optimize individual neural control of behavior across a large population of candidate body plans and return a fitness score for each design based on the performance of its fully optimized behavior. Non-differentiable changes to the mechanical structure of each robot in the population -- mutations that rearrange, combine, add, or remove body parts -- were applied by a genetic algorithm in an outer loop of search, generating a continuous flow of novel morphologies with highly-coordinated and graceful behaviors honed by gradient descent. This enabled the exploration of several orders-of-magnitude more designs than all previous methods, despite the fact that robots here have the potential to be much more complex, in terms of number of independent motors, than those in prior studies. We found that evolution reliably produces ``increasingly differentiable'' robots: body plans that smooth the loss landscape in which learning operates and thereby provide better training paths toward performant behaviors. Finally, one of the highly differentiable morphologies discovered in simulation was realized as a physical robot and shown to retain its optimized behavior. This provides a cyberphysical platform to investigate the relationship between evolution and learning in biological systems and broadens our understanding of how a robot's physical structure can influence the ability to train policies for it. Videos and code at https://sites.google.com/view/eldir.</description><identifier>DOI: 10.48550/arxiv.2405.14712</identifier><language>eng</language><subject>Computer Science - Artificial Intelligence ; Computer Science - Robotics</subject><creationdate>2024-05</creationdate><rights>http://creativecommons.org/licenses/by/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2405.14712$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2405.14712$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Strgar, Luke</creatorcontrib><creatorcontrib>Matthews, David</creatorcontrib><creatorcontrib>Hummer, Tyler</creatorcontrib><creatorcontrib>Kriegman, Sam</creatorcontrib><title>Evolution and learning in differentiable robots</title><description>The automatic design of robots has existed for 30 years but has been constricted by serial non-differentiable design evaluations, premature convergence to simple bodies or clumsy behaviors, and a lack of sim2real transfer to physical machines. Thus, here we employ massively-parallel differentiable simulations to rapidly and simultaneously optimize individual neural control of behavior across a large population of candidate body plans and return a fitness score for each design based on the performance of its fully optimized behavior. Non-differentiable changes to the mechanical structure of each robot in the population -- mutations that rearrange, combine, add, or remove body parts -- were applied by a genetic algorithm in an outer loop of search, generating a continuous flow of novel morphologies with highly-coordinated and graceful behaviors honed by gradient descent. This enabled the exploration of several orders-of-magnitude more designs than all previous methods, despite the fact that robots here have the potential to be much more complex, in terms of number of independent motors, than those in prior studies. We found that evolution reliably produces ``increasingly differentiable'' robots: body plans that smooth the loss landscape in which learning operates and thereby provide better training paths toward performant behaviors. Finally, one of the highly differentiable morphologies discovered in simulation was realized as a physical robot and shown to retain its optimized behavior. This provides a cyberphysical platform to investigate the relationship between evolution and learning in biological systems and broadens our understanding of how a robot's physical structure can influence the ability to train policies for it. Videos and code at https://sites.google.com/view/eldir.</description><subject>Computer Science - Artificial Intelligence</subject><subject>Computer Science - Robotics</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotzrsKwjAUgOEsDqI-gJN5gdbc2rSjlHoBwcW9nOYkEqippFX07cXL9G8_HyFLzlJVZBlbQ3z6RyoUy1KuNBdTsq4ffXcffR8oBKSdhRh8uFAfKHrnbLRh9NB2lsa-7cdhTiYOusEu_p2R87Y-V_vkeNodqs0xgVyLxJXYMoEaDbOIIEXOhJSqAODCYCEN5MgkohIAkKlCq1K3aIzjhmuDRs7I6rf9iptb9FeIr-Yjb75y-QZYgT7l</recordid><startdate>20240523</startdate><enddate>20240523</enddate><creator>Strgar, Luke</creator><creator>Matthews, David</creator><creator>Hummer, Tyler</creator><creator>Kriegman, Sam</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20240523</creationdate><title>Evolution and learning in differentiable robots</title><author>Strgar, Luke ; Matthews, David ; Hummer, Tyler ; Kriegman, Sam</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a672-f9db02d7dc0edda326023348aa12cd83ca6d03dd42aaa5487497bdccf1c17cdc3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Computer Science - Artificial Intelligence</topic><topic>Computer Science - Robotics</topic><toplevel>online_resources</toplevel><creatorcontrib>Strgar, Luke</creatorcontrib><creatorcontrib>Matthews, David</creatorcontrib><creatorcontrib>Hummer, Tyler</creatorcontrib><creatorcontrib>Kriegman, Sam</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Strgar, Luke</au><au>Matthews, David</au><au>Hummer, Tyler</au><au>Kriegman, Sam</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Evolution and learning in differentiable robots</atitle><date>2024-05-23</date><risdate>2024</risdate><abstract>The automatic design of robots has existed for 30 years but has been constricted by serial non-differentiable design evaluations, premature convergence to simple bodies or clumsy behaviors, and a lack of sim2real transfer to physical machines. Thus, here we employ massively-parallel differentiable simulations to rapidly and simultaneously optimize individual neural control of behavior across a large population of candidate body plans and return a fitness score for each design based on the performance of its fully optimized behavior. Non-differentiable changes to the mechanical structure of each robot in the population -- mutations that rearrange, combine, add, or remove body parts -- were applied by a genetic algorithm in an outer loop of search, generating a continuous flow of novel morphologies with highly-coordinated and graceful behaviors honed by gradient descent. This enabled the exploration of several orders-of-magnitude more designs than all previous methods, despite the fact that robots here have the potential to be much more complex, in terms of number of independent motors, than those in prior studies. We found that evolution reliably produces ``increasingly differentiable'' robots: body plans that smooth the loss landscape in which learning operates and thereby provide better training paths toward performant behaviors. Finally, one of the highly differentiable morphologies discovered in simulation was realized as a physical robot and shown to retain its optimized behavior. This provides a cyberphysical platform to investigate the relationship between evolution and learning in biological systems and broadens our understanding of how a robot's physical structure can influence the ability to train policies for it. Videos and code at https://sites.google.com/view/eldir.</abstract><doi>10.48550/arxiv.2405.14712</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2405.14712
ispartof
issn
language eng
recordid cdi_arxiv_primary_2405_14712
source arXiv.org
subjects Computer Science - Artificial Intelligence
Computer Science - Robotics
title Evolution and learning in differentiable robots
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T10%3A58%3A37IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Evolution%20and%20learning%20in%20differentiable%20robots&rft.au=Strgar,%20Luke&rft.date=2024-05-23&rft_id=info:doi/10.48550/arxiv.2405.14712&rft_dat=%3Carxiv_GOX%3E2405_14712%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true