BARK: Open Behavior Benchmarking in Multi-Agent Environments

Predicting and planning interactive behaviors in complex traffic situations presents a challenging task. Especially in scenarios involving multiple traffic participants that interact densely, autonomous vehicles still struggle to interpret situations and to eventually achieve their own mission goal....

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:arXiv.org 2020-09
Hauptverfasser: Bernhard, Julian, Esterle, Klemens, Hart, Patrick, Kessler, Tobias
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Bernhard, Julian
Esterle, Klemens
Hart, Patrick
Kessler, Tobias
description Predicting and planning interactive behaviors in complex traffic situations presents a challenging task. Especially in scenarios involving multiple traffic participants that interact densely, autonomous vehicles still struggle to interpret situations and to eventually achieve their own mission goal. As driving tests are costly and challenging scenarios are hard to find and reproduce, simulation is widely used to develop, test, and benchmark behavior models. However, most simulations rely on datasets and simplistic behavior models for traffic participants and do not cover the full variety of real-world, interactive human behaviors. In this work, we introduce BARK, an open-source behavior benchmarking environment designed to mitigate the shortcomings stated above. In BARK, behavior models are (re-)used for planning, prediction, and simulation. A range of models is currently available, such as Monte-Carlo Tree Search and Reinforcement Learning-based behavior models. We use a public dataset and sampling-based scenario generation to show the inter-exchangeability of behavior models in BARK. We evaluate how well the models used cope with interactions and how robust they are towards exchanging behavior models. Our evaluation shows that BARK provides a suitable framework for a systematic development of behavior models.
doi_str_mv 10.48550/arxiv.2003.02604
format Article
fullrecord <record><control><sourceid>proquest_arxiv</sourceid><recordid>TN_cdi_arxiv_primary_2003_02604</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2372429307</sourcerecordid><originalsourceid>FETCH-LOGICAL-a527-f461306638488802dd5f99bf9131c4eea5d867c4b02f31033b20e35ae4a76cfe3</originalsourceid><addsrcrecordid>eNotj1tLw0AUhBdBsNT-AJ8M-Jx4cs5ushFf0lIvWClI38Mm2bRb203cXNB_b2x9moEZhvkYuwkh4FIIuFfu2wwBAlAAGAG_YBMkCn3JEa_YrG33AGMQoxA0YY_z9OPtwVs32npzvVODqd1obLE7Kvdp7NYz1nvvD53x0622nbe0g3G1PY6-vWaXlTq0evavU7Z5Wm4WL_5q_fy6SFe-Ehj7FY9CgigiyaWUgGUpqiTJqySksOBaK1HKKC54DlhRCEQ5giahNFdxVFSapuz2PHtCyxpnxm8_2R9idkIcG3fnRuPqr163Xbave2fHTxlSjBwTgph-ATnNUdM</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2372429307</pqid></control><display><type>article</type><title>BARK: Open Behavior Benchmarking in Multi-Agent Environments</title><source>Freely Accessible Journals</source><source>arXiv.org</source><creator>Bernhard, Julian ; Esterle, Klemens ; Hart, Patrick ; Kessler, Tobias</creator><creatorcontrib>Bernhard, Julian ; Esterle, Klemens ; Hart, Patrick ; Kessler, Tobias</creatorcontrib><description>Predicting and planning interactive behaviors in complex traffic situations presents a challenging task. Especially in scenarios involving multiple traffic participants that interact densely, autonomous vehicles still struggle to interpret situations and to eventually achieve their own mission goal. As driving tests are costly and challenging scenarios are hard to find and reproduce, simulation is widely used to develop, test, and benchmark behavior models. However, most simulations rely on datasets and simplistic behavior models for traffic participants and do not cover the full variety of real-world, interactive human behaviors. In this work, we introduce BARK, an open-source behavior benchmarking environment designed to mitigate the shortcomings stated above. In BARK, behavior models are (re-)used for planning, prediction, and simulation. A range of models is currently available, such as Monte-Carlo Tree Search and Reinforcement Learning-based behavior models. We use a public dataset and sampling-based scenario generation to show the inter-exchangeability of behavior models in BARK. We evaluate how well the models used cope with interactions and how robust they are towards exchanging behavior models. Our evaluation shows that BARK provides a suitable framework for a systematic development of behavior models.</description><identifier>EISSN: 2331-8422</identifier><identifier>DOI: 10.48550/arxiv.2003.02604</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Behavior ; Benchmarks ; Computer Science - Multiagent Systems ; Computer simulation ; Datasets ; Driving ; Monte Carlo simulation ; Multiagent systems ; Traffic models ; Traffic planning</subject><ispartof>arXiv.org, 2020-09</ispartof><rights>2020. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,784,885,27925</link.rule.ids><backlink>$$Uhttps://doi.org/10.1109/IROS45743.2020.9341222$$DView published paper (Access to full text may be restricted)$$Hfree_for_read</backlink><backlink>$$Uhttps://doi.org/10.48550/arXiv.2003.02604$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Bernhard, Julian</creatorcontrib><creatorcontrib>Esterle, Klemens</creatorcontrib><creatorcontrib>Hart, Patrick</creatorcontrib><creatorcontrib>Kessler, Tobias</creatorcontrib><title>BARK: Open Behavior Benchmarking in Multi-Agent Environments</title><title>arXiv.org</title><description>Predicting and planning interactive behaviors in complex traffic situations presents a challenging task. Especially in scenarios involving multiple traffic participants that interact densely, autonomous vehicles still struggle to interpret situations and to eventually achieve their own mission goal. As driving tests are costly and challenging scenarios are hard to find and reproduce, simulation is widely used to develop, test, and benchmark behavior models. However, most simulations rely on datasets and simplistic behavior models for traffic participants and do not cover the full variety of real-world, interactive human behaviors. In this work, we introduce BARK, an open-source behavior benchmarking environment designed to mitigate the shortcomings stated above. In BARK, behavior models are (re-)used for planning, prediction, and simulation. A range of models is currently available, such as Monte-Carlo Tree Search and Reinforcement Learning-based behavior models. We use a public dataset and sampling-based scenario generation to show the inter-exchangeability of behavior models in BARK. We evaluate how well the models used cope with interactions and how robust they are towards exchanging behavior models. Our evaluation shows that BARK provides a suitable framework for a systematic development of behavior models.</description><subject>Behavior</subject><subject>Benchmarks</subject><subject>Computer Science - Multiagent Systems</subject><subject>Computer simulation</subject><subject>Datasets</subject><subject>Driving</subject><subject>Monte Carlo simulation</subject><subject>Multiagent systems</subject><subject>Traffic models</subject><subject>Traffic planning</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GOX</sourceid><recordid>eNotj1tLw0AUhBdBsNT-AJ8M-Jx4cs5ushFf0lIvWClI38Mm2bRb203cXNB_b2x9moEZhvkYuwkh4FIIuFfu2wwBAlAAGAG_YBMkCn3JEa_YrG33AGMQoxA0YY_z9OPtwVs32npzvVODqd1obLE7Kvdp7NYz1nvvD53x0622nbe0g3G1PY6-vWaXlTq0evavU7Z5Wm4WL_5q_fy6SFe-Ehj7FY9CgigiyaWUgGUpqiTJqySksOBaK1HKKC54DlhRCEQ5giahNFdxVFSapuz2PHtCyxpnxm8_2R9idkIcG3fnRuPqr163Xbave2fHTxlSjBwTgph-ATnNUdM</recordid><startdate>20200929</startdate><enddate>20200929</enddate><creator>Bernhard, Julian</creator><creator>Esterle, Klemens</creator><creator>Hart, Patrick</creator><creator>Kessler, Tobias</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PTHSS</scope><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20200929</creationdate><title>BARK: Open Behavior Benchmarking in Multi-Agent Environments</title><author>Bernhard, Julian ; Esterle, Klemens ; Hart, Patrick ; Kessler, Tobias</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a527-f461306638488802dd5f99bf9131c4eea5d867c4b02f31033b20e35ae4a76cfe3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Behavior</topic><topic>Benchmarks</topic><topic>Computer Science - Multiagent Systems</topic><topic>Computer simulation</topic><topic>Datasets</topic><topic>Driving</topic><topic>Monte Carlo simulation</topic><topic>Multiagent systems</topic><topic>Traffic models</topic><topic>Traffic planning</topic><toplevel>online_resources</toplevel><creatorcontrib>Bernhard, Julian</creatorcontrib><creatorcontrib>Esterle, Klemens</creatorcontrib><creatorcontrib>Hart, Patrick</creatorcontrib><creatorcontrib>Kessler, Tobias</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>Engineering Collection</collection><collection>arXiv Computer Science</collection><collection>arXiv.org</collection><jtitle>arXiv.org</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Bernhard, Julian</au><au>Esterle, Klemens</au><au>Hart, Patrick</au><au>Kessler, Tobias</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>BARK: Open Behavior Benchmarking in Multi-Agent Environments</atitle><jtitle>arXiv.org</jtitle><date>2020-09-29</date><risdate>2020</risdate><eissn>2331-8422</eissn><abstract>Predicting and planning interactive behaviors in complex traffic situations presents a challenging task. Especially in scenarios involving multiple traffic participants that interact densely, autonomous vehicles still struggle to interpret situations and to eventually achieve their own mission goal. As driving tests are costly and challenging scenarios are hard to find and reproduce, simulation is widely used to develop, test, and benchmark behavior models. However, most simulations rely on datasets and simplistic behavior models for traffic participants and do not cover the full variety of real-world, interactive human behaviors. In this work, we introduce BARK, an open-source behavior benchmarking environment designed to mitigate the shortcomings stated above. In BARK, behavior models are (re-)used for planning, prediction, and simulation. A range of models is currently available, such as Monte-Carlo Tree Search and Reinforcement Learning-based behavior models. We use a public dataset and sampling-based scenario generation to show the inter-exchangeability of behavior models in BARK. We evaluate how well the models used cope with interactions and how robust they are towards exchanging behavior models. Our evaluation shows that BARK provides a suitable framework for a systematic development of behavior models.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><doi>10.48550/arxiv.2003.02604</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2020-09
issn 2331-8422
language eng
recordid cdi_arxiv_primary_2003_02604
source Freely Accessible Journals; arXiv.org
subjects Behavior
Benchmarks
Computer Science - Multiagent Systems
Computer simulation
Datasets
Driving
Monte Carlo simulation
Multiagent systems
Traffic models
Traffic planning
title BARK: Open Behavior Benchmarking in Multi-Agent Environments
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-25T23%3A02%3A34IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_arxiv&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=BARK:%20Open%20Behavior%20Benchmarking%20in%20Multi-Agent%20Environments&rft.jtitle=arXiv.org&rft.au=Bernhard,%20Julian&rft.date=2020-09-29&rft.eissn=2331-8422&rft_id=info:doi/10.48550/arxiv.2003.02604&rft_dat=%3Cproquest_arxiv%3E2372429307%3C/proquest_arxiv%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2372429307&rft_id=info:pmid/&rfr_iscdi=true