Driver-like decision-making method for vehicle longitudinal autonomous driving based on deep reinforcement learning

Decision-making is one of the key parts of the research on vehicle longitudinal autonomous driving. Considering the behavior of human drivers when designing autonomous driving decision-making strategies is a current research hotspot. In longitudinal autonomous driving decision-making strategies, tra...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings of the Institution of Mechanical Engineers. Part D, Journal of automobile engineering Journal of automobile engineering, 2022-11, Vol.236 (13), p.3060-3070
Hauptverfasser: Gao, Zhenhai, Yan, Xiangtong, Gao, Fei, He, Lei
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 3070
container_issue 13
container_start_page 3060
container_title Proceedings of the Institution of Mechanical Engineers. Part D, Journal of automobile engineering
container_volume 236
creator Gao, Zhenhai
Yan, Xiangtong
Gao, Fei
He, Lei
description Decision-making is one of the key parts of the research on vehicle longitudinal autonomous driving. Considering the behavior of human drivers when designing autonomous driving decision-making strategies is a current research hotspot. In longitudinal autonomous driving decision-making strategies, traditional rule-based decision-making strategies are difficult to apply to complex scenarios. Current decision-making methods that use reinforcement learning and deep reinforcement learning construct reward functions designed with safety, comfort, and economy. Compared with human drivers, the obtained decision strategies still have big gaps. Focusing on the above problems, this paper uses the driver’s behavior data to design the reward function of the deep reinforcement learning algorithm through BP neural network fitting, and uses the deep reinforcement learning DQN algorithm and the DDPG algorithm to establish two driver-like longitudinal autonomous driving decision-making models. The simulation experiment compares the decision-making effect of the two models with the driver curve. The results shows that the two algorithms can realize driver-like decision-making, and the consistency of the DDPG algorithm and human driver behavior is higher than that of the DQN algorithm, the effect of the DDPG algorithm is better than the DQN algorithm.
doi_str_mv 10.1177/09544070211063081
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2718094421</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sage_id>10.1177_09544070211063081</sage_id><sourcerecordid>2718094421</sourcerecordid><originalsourceid>FETCH-LOGICAL-c312t-b97b0e256d836c506b34c53e6af91a8da8053de6cd72fd37aa655a1765916d403</originalsourceid><addsrcrecordid>eNp1kMtOwzAQRS0EEuXxAewssU6xHT-SJSpPqRIbWEeuPWndJnaxk0r8PY6KxAIxm1nMOVeji9ANJXNKlbojteCcKMIoJbIkFT1BM0Y4LVhd01M0m-7FBJyji5S2JI_iYobSQ3QHiEXndoAtGJdc8EWvd86vcQ_DJljchogPsHGmA9wFv3bDaJ3XHdbjEHzow5iwzTGTstIJLA4-Z8EeR3A-2wZ68APuQEefoSt01uouwfXPvkQfT4_vi5di-fb8urhfFqakbChWtVoRYELaqpRGELkquRElSN3WVFdWV0SUFqSxirW2VFpLITRVUtRUWk7KS3R7zN3H8DlCGpptGGN-PDVM0YrUnDOaKXqkTAwpRWibfXS9jl8NJc3UbfOn2-zMj07Sa_hN_V_4BqCgeqo</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2718094421</pqid></control><display><type>article</type><title>Driver-like decision-making method for vehicle longitudinal autonomous driving based on deep reinforcement learning</title><source>SAGE Complete</source><creator>Gao, Zhenhai ; Yan, Xiangtong ; Gao, Fei ; He, Lei</creator><creatorcontrib>Gao, Zhenhai ; Yan, Xiangtong ; Gao, Fei ; He, Lei</creatorcontrib><description>Decision-making is one of the key parts of the research on vehicle longitudinal autonomous driving. Considering the behavior of human drivers when designing autonomous driving decision-making strategies is a current research hotspot. In longitudinal autonomous driving decision-making strategies, traditional rule-based decision-making strategies are difficult to apply to complex scenarios. Current decision-making methods that use reinforcement learning and deep reinforcement learning construct reward functions designed with safety, comfort, and economy. Compared with human drivers, the obtained decision strategies still have big gaps. Focusing on the above problems, this paper uses the driver’s behavior data to design the reward function of the deep reinforcement learning algorithm through BP neural network fitting, and uses the deep reinforcement learning DQN algorithm and the DDPG algorithm to establish two driver-like longitudinal autonomous driving decision-making models. The simulation experiment compares the decision-making effect of the two models with the driver curve. The results shows that the two algorithms can realize driver-like decision-making, and the consistency of the DDPG algorithm and human driver behavior is higher than that of the DQN algorithm, the effect of the DDPG algorithm is better than the DQN algorithm.</description><identifier>ISSN: 0954-4070</identifier><identifier>EISSN: 2041-2991</identifier><identifier>DOI: 10.1177/09544070211063081</identifier><language>eng</language><publisher>London, England: SAGE Publications</publisher><subject>Algorithms ; Back propagation networks ; Decision making ; Deep learning ; Driver behavior ; Machine learning</subject><ispartof>Proceedings of the Institution of Mechanical Engineers. Part D, Journal of automobile engineering, 2022-11, Vol.236 (13), p.3060-3070</ispartof><rights>IMechE 2021</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c312t-b97b0e256d836c506b34c53e6af91a8da8053de6cd72fd37aa655a1765916d403</citedby><cites>FETCH-LOGICAL-c312t-b97b0e256d836c506b34c53e6af91a8da8053de6cd72fd37aa655a1765916d403</cites><orcidid>0000-0003-2674-916X ; 0000-0003-4195-5033</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://journals.sagepub.com/doi/pdf/10.1177/09544070211063081$$EPDF$$P50$$Gsage$$H</linktopdf><linktohtml>$$Uhttps://journals.sagepub.com/doi/10.1177/09544070211063081$$EHTML$$P50$$Gsage$$H</linktohtml><link.rule.ids>314,776,780,21798,27901,27902,43597,43598</link.rule.ids></links><search><creatorcontrib>Gao, Zhenhai</creatorcontrib><creatorcontrib>Yan, Xiangtong</creatorcontrib><creatorcontrib>Gao, Fei</creatorcontrib><creatorcontrib>He, Lei</creatorcontrib><title>Driver-like decision-making method for vehicle longitudinal autonomous driving based on deep reinforcement learning</title><title>Proceedings of the Institution of Mechanical Engineers. Part D, Journal of automobile engineering</title><description>Decision-making is one of the key parts of the research on vehicle longitudinal autonomous driving. Considering the behavior of human drivers when designing autonomous driving decision-making strategies is a current research hotspot. In longitudinal autonomous driving decision-making strategies, traditional rule-based decision-making strategies are difficult to apply to complex scenarios. Current decision-making methods that use reinforcement learning and deep reinforcement learning construct reward functions designed with safety, comfort, and economy. Compared with human drivers, the obtained decision strategies still have big gaps. Focusing on the above problems, this paper uses the driver’s behavior data to design the reward function of the deep reinforcement learning algorithm through BP neural network fitting, and uses the deep reinforcement learning DQN algorithm and the DDPG algorithm to establish two driver-like longitudinal autonomous driving decision-making models. The simulation experiment compares the decision-making effect of the two models with the driver curve. The results shows that the two algorithms can realize driver-like decision-making, and the consistency of the DDPG algorithm and human driver behavior is higher than that of the DQN algorithm, the effect of the DDPG algorithm is better than the DQN algorithm.</description><subject>Algorithms</subject><subject>Back propagation networks</subject><subject>Decision making</subject><subject>Deep learning</subject><subject>Driver behavior</subject><subject>Machine learning</subject><issn>0954-4070</issn><issn>2041-2991</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><recordid>eNp1kMtOwzAQRS0EEuXxAewssU6xHT-SJSpPqRIbWEeuPWndJnaxk0r8PY6KxAIxm1nMOVeji9ANJXNKlbojteCcKMIoJbIkFT1BM0Y4LVhd01M0m-7FBJyji5S2JI_iYobSQ3QHiEXndoAtGJdc8EWvd86vcQ_DJljchogPsHGmA9wFv3bDaJ3XHdbjEHzow5iwzTGTstIJLA4-Z8EeR3A-2wZ68APuQEefoSt01uouwfXPvkQfT4_vi5di-fb8urhfFqakbChWtVoRYELaqpRGELkquRElSN3WVFdWV0SUFqSxirW2VFpLITRVUtRUWk7KS3R7zN3H8DlCGpptGGN-PDVM0YrUnDOaKXqkTAwpRWibfXS9jl8NJc3UbfOn2-zMj07Sa_hN_V_4BqCgeqo</recordid><startdate>20221101</startdate><enddate>20221101</enddate><creator>Gao, Zhenhai</creator><creator>Yan, Xiangtong</creator><creator>Gao, Fei</creator><creator>He, Lei</creator><general>SAGE Publications</general><general>SAGE PUBLICATIONS, INC</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7TB</scope><scope>8FD</scope><scope>F28</scope><scope>FR3</scope><orcidid>https://orcid.org/0000-0003-2674-916X</orcidid><orcidid>https://orcid.org/0000-0003-4195-5033</orcidid></search><sort><creationdate>20221101</creationdate><title>Driver-like decision-making method for vehicle longitudinal autonomous driving based on deep reinforcement learning</title><author>Gao, Zhenhai ; Yan, Xiangtong ; Gao, Fei ; He, Lei</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c312t-b97b0e256d836c506b34c53e6af91a8da8053de6cd72fd37aa655a1765916d403</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Algorithms</topic><topic>Back propagation networks</topic><topic>Decision making</topic><topic>Deep learning</topic><topic>Driver behavior</topic><topic>Machine learning</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Gao, Zhenhai</creatorcontrib><creatorcontrib>Yan, Xiangtong</creatorcontrib><creatorcontrib>Gao, Fei</creatorcontrib><creatorcontrib>He, Lei</creatorcontrib><collection>CrossRef</collection><collection>Mechanical &amp; Transportation Engineering Abstracts</collection><collection>Technology Research Database</collection><collection>ANTE: Abstracts in New Technology &amp; Engineering</collection><collection>Engineering Research Database</collection><jtitle>Proceedings of the Institution of Mechanical Engineers. Part D, Journal of automobile engineering</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Gao, Zhenhai</au><au>Yan, Xiangtong</au><au>Gao, Fei</au><au>He, Lei</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Driver-like decision-making method for vehicle longitudinal autonomous driving based on deep reinforcement learning</atitle><jtitle>Proceedings of the Institution of Mechanical Engineers. Part D, Journal of automobile engineering</jtitle><date>2022-11-01</date><risdate>2022</risdate><volume>236</volume><issue>13</issue><spage>3060</spage><epage>3070</epage><pages>3060-3070</pages><issn>0954-4070</issn><eissn>2041-2991</eissn><abstract>Decision-making is one of the key parts of the research on vehicle longitudinal autonomous driving. Considering the behavior of human drivers when designing autonomous driving decision-making strategies is a current research hotspot. In longitudinal autonomous driving decision-making strategies, traditional rule-based decision-making strategies are difficult to apply to complex scenarios. Current decision-making methods that use reinforcement learning and deep reinforcement learning construct reward functions designed with safety, comfort, and economy. Compared with human drivers, the obtained decision strategies still have big gaps. Focusing on the above problems, this paper uses the driver’s behavior data to design the reward function of the deep reinforcement learning algorithm through BP neural network fitting, and uses the deep reinforcement learning DQN algorithm and the DDPG algorithm to establish two driver-like longitudinal autonomous driving decision-making models. The simulation experiment compares the decision-making effect of the two models with the driver curve. The results shows that the two algorithms can realize driver-like decision-making, and the consistency of the DDPG algorithm and human driver behavior is higher than that of the DQN algorithm, the effect of the DDPG algorithm is better than the DQN algorithm.</abstract><cop>London, England</cop><pub>SAGE Publications</pub><doi>10.1177/09544070211063081</doi><tpages>11</tpages><orcidid>https://orcid.org/0000-0003-2674-916X</orcidid><orcidid>https://orcid.org/0000-0003-4195-5033</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 0954-4070
ispartof Proceedings of the Institution of Mechanical Engineers. Part D, Journal of automobile engineering, 2022-11, Vol.236 (13), p.3060-3070
issn 0954-4070
2041-2991
language eng
recordid cdi_proquest_journals_2718094421
source SAGE Complete
subjects Algorithms
Back propagation networks
Decision making
Deep learning
Driver behavior
Machine learning
title Driver-like decision-making method for vehicle longitudinal autonomous driving based on deep reinforcement learning
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-30T16%3A09%3A50IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Driver-like%20decision-making%20method%20for%20vehicle%20longitudinal%20autonomous%20driving%20based%20on%20deep%20reinforcement%20learning&rft.jtitle=Proceedings%20of%20the%20Institution%20of%20Mechanical%20Engineers.%20Part%20D,%20Journal%20of%20automobile%20engineering&rft.au=Gao,%20Zhenhai&rft.date=2022-11-01&rft.volume=236&rft.issue=13&rft.spage=3060&rft.epage=3070&rft.pages=3060-3070&rft.issn=0954-4070&rft.eissn=2041-2991&rft_id=info:doi/10.1177/09544070211063081&rft_dat=%3Cproquest_cross%3E2718094421%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2718094421&rft_id=info:pmid/&rft_sage_id=10.1177_09544070211063081&rfr_iscdi=true