Distributed agent-based deep reinforcement learning for large scale traffic signal control
Traffic signal control (TSC) is an established yet challenging engineering solution that alleviates traffic congestion by coordinating vehicles’ movements at road intersections. Theoretically, reinforcement learning (RL) is a promising method for adaptive TSC in complex urban traffic networks. Howev...
Gespeichert in:
| Veröffentlicht in: | Knowledge-based systems 2022-04, Vol.241, p.108304, Article 108304 |
|---|---|
| Hauptverfasser: | , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | 108304 |
| container_title | Knowledge-based systems |
| container_volume | 241 |
| creator | Wu, Qiang Wu, Jianqing Shen, Jun Du, Bo Telikani, Akbar Fahmideh, Mahdi Liang, Chao |
| description | Traffic signal control (TSC) is an established yet challenging engineering solution that alleviates traffic congestion by coordinating vehicles’ movements at road intersections. Theoretically, reinforcement learning (RL) is a promising method for adaptive TSC in complex urban traffic networks. However, current TSC systems still rely heavily on simplified rule-based methods in practice. In this paper, we propose: (1) two game theory-aided RL algorithms leveraging Nash Equilibrium and RL, namely Nash Advantage Actor–Critic (Nash-A2C) and Nash Asynchronous Advantage Actor–Critic (Nash-A3C); (2) a distributed computing Internet of Things (IoT) architecture for traffic simulation, which is more suitable for distributed TSC methods like the Nash-A3C deployment in its fog layer. We apply both methods in our computing architecture and obtain better performance than benchmark TSC methods by 22.1% and 9.7% reduction of congestion time and network delay, respectively. |
| doi_str_mv | 10.1016/j.knosys.2022.108304 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2642939652</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S095070512200106X</els_id><sourcerecordid>2642939652</sourcerecordid><originalsourceid>FETCH-LOGICAL-c264t-ff0738ac360ae09c6fb7c3185170ec9040ccacb0dc6e788352945150f80127753</originalsourceid><addsrcrecordid>eNp9UMtOwzAQtBBIlMIfcLDEOWXtPJxckFB5SpW4wIWL5WzWkUOaFDtF6t_jKpw57Wp2ZnZ3GLsWsBIgittu9TWM4RBWEqSMUJlCdsIWolQyURlUp2wBVQ6Jglycs4sQOoDIFOWCfT64MHlX7ydquGlpmJLahNg3RDvuyQ129EjbOOA9GT-4oeUR4r3xLfGApic-eWOtQx5cO5ie4zhMfuwv2Zk1faCrv7pkH0-P7-uXZPP2_Lq-3yQoi2xKrAWVlgbTAgxBhYWtFaaizIUCwgoyQDRYQ4MFqbJMc1llucjBliCkUnm6ZDez786P33sKk-7GvY-HBB0XyCqtilxGVjaz0I8heLJ6593W-IMWoI8p6k7PKepjinpOMcruZhnFD34ceR3Q0YDUOE846WZ0_xv8AiFOfVY</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2642939652</pqid></control><display><type>article</type><title>Distributed agent-based deep reinforcement learning for large scale traffic signal control</title><source>Access via ScienceDirect (Elsevier)</source><creator>Wu, Qiang ; Wu, Jianqing ; Shen, Jun ; Du, Bo ; Telikani, Akbar ; Fahmideh, Mahdi ; Liang, Chao</creator><creatorcontrib>Wu, Qiang ; Wu, Jianqing ; Shen, Jun ; Du, Bo ; Telikani, Akbar ; Fahmideh, Mahdi ; Liang, Chao</creatorcontrib><description>Traffic signal control (TSC) is an established yet challenging engineering solution that alleviates traffic congestion by coordinating vehicles’ movements at road intersections. Theoretically, reinforcement learning (RL) is a promising method for adaptive TSC in complex urban traffic networks. However, current TSC systems still rely heavily on simplified rule-based methods in practice. In this paper, we propose: (1) two game theory-aided RL algorithms leveraging Nash Equilibrium and RL, namely Nash Advantage Actor–Critic (Nash-A2C) and Nash Asynchronous Advantage Actor–Critic (Nash-A3C); (2) a distributed computing Internet of Things (IoT) architecture for traffic simulation, which is more suitable for distributed TSC methods like the Nash-A3C deployment in its fog layer. We apply both methods in our computing architecture and obtain better performance than benchmark TSC methods by 22.1% and 9.7% reduction of congestion time and network delay, respectively.</description><identifier>ISSN: 0950-7051</identifier><identifier>EISSN: 1872-7409</identifier><identifier>DOI: 10.1016/j.knosys.2022.108304</identifier><language>eng</language><publisher>Amsterdam: Elsevier B.V</publisher><subject>Algorithms ; Computer architecture ; Computer networks ; Deep learning ; Distributed computing architecture ; Distributed processing ; Game theory ; Internet of Things ; Machine learning ; Nash Equilibrium ; Nash-A3C ; Reinforcement learning ; Traffic congestion ; Traffic control ; Traffic engineering ; Traffic signal control ; Traffic signals</subject><ispartof>Knowledge-based systems, 2022-04, Vol.241, p.108304, Article 108304</ispartof><rights>2022 Elsevier B.V.</rights><rights>Copyright Elsevier Science Ltd. Apr 6, 2022</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c264t-ff0738ac360ae09c6fb7c3185170ec9040ccacb0dc6e788352945150f80127753</citedby><cites>FETCH-LOGICAL-c264t-ff0738ac360ae09c6fb7c3185170ec9040ccacb0dc6e788352945150f80127753</cites><orcidid>0000-0002-9403-7140 ; 0000-0003-4467-4915 ; 0000-0003-0655-0479 ; 0000-0001-7198-4199</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://dx.doi.org/10.1016/j.knosys.2022.108304$$EHTML$$P50$$Gelsevier$$H</linktohtml><link.rule.ids>315,781,785,3551,27929,27930,46000</link.rule.ids></links><search><creatorcontrib>Wu, Qiang</creatorcontrib><creatorcontrib>Wu, Jianqing</creatorcontrib><creatorcontrib>Shen, Jun</creatorcontrib><creatorcontrib>Du, Bo</creatorcontrib><creatorcontrib>Telikani, Akbar</creatorcontrib><creatorcontrib>Fahmideh, Mahdi</creatorcontrib><creatorcontrib>Liang, Chao</creatorcontrib><title>Distributed agent-based deep reinforcement learning for large scale traffic signal control</title><title>Knowledge-based systems</title><description>Traffic signal control (TSC) is an established yet challenging engineering solution that alleviates traffic congestion by coordinating vehicles’ movements at road intersections. Theoretically, reinforcement learning (RL) is a promising method for adaptive TSC in complex urban traffic networks. However, current TSC systems still rely heavily on simplified rule-based methods in practice. In this paper, we propose: (1) two game theory-aided RL algorithms leveraging Nash Equilibrium and RL, namely Nash Advantage Actor–Critic (Nash-A2C) and Nash Asynchronous Advantage Actor–Critic (Nash-A3C); (2) a distributed computing Internet of Things (IoT) architecture for traffic simulation, which is more suitable for distributed TSC methods like the Nash-A3C deployment in its fog layer. We apply both methods in our computing architecture and obtain better performance than benchmark TSC methods by 22.1% and 9.7% reduction of congestion time and network delay, respectively.</description><subject>Algorithms</subject><subject>Computer architecture</subject><subject>Computer networks</subject><subject>Deep learning</subject><subject>Distributed computing architecture</subject><subject>Distributed processing</subject><subject>Game theory</subject><subject>Internet of Things</subject><subject>Machine learning</subject><subject>Nash Equilibrium</subject><subject>Nash-A3C</subject><subject>Reinforcement learning</subject><subject>Traffic congestion</subject><subject>Traffic control</subject><subject>Traffic engineering</subject><subject>Traffic signal control</subject><subject>Traffic signals</subject><issn>0950-7051</issn><issn>1872-7409</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><recordid>eNp9UMtOwzAQtBBIlMIfcLDEOWXtPJxckFB5SpW4wIWL5WzWkUOaFDtF6t_jKpw57Wp2ZnZ3GLsWsBIgittu9TWM4RBWEqSMUJlCdsIWolQyURlUp2wBVQ6Jglycs4sQOoDIFOWCfT64MHlX7ydquGlpmJLahNg3RDvuyQ129EjbOOA9GT-4oeUR4r3xLfGApic-eWOtQx5cO5ie4zhMfuwv2Zk1faCrv7pkH0-P7-uXZPP2_Lq-3yQoi2xKrAWVlgbTAgxBhYWtFaaizIUCwgoyQDRYQ4MFqbJMc1llucjBliCkUnm6ZDez786P33sKk-7GvY-HBB0XyCqtilxGVjaz0I8heLJ6593W-IMWoI8p6k7PKepjinpOMcruZhnFD34ceR3Q0YDUOE846WZ0_xv8AiFOfVY</recordid><startdate>20220406</startdate><enddate>20220406</enddate><creator>Wu, Qiang</creator><creator>Wu, Jianqing</creator><creator>Shen, Jun</creator><creator>Du, Bo</creator><creator>Telikani, Akbar</creator><creator>Fahmideh, Mahdi</creator><creator>Liang, Chao</creator><general>Elsevier B.V</general><general>Elsevier Science Ltd</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>8FD</scope><scope>E3H</scope><scope>F2A</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0002-9403-7140</orcidid><orcidid>https://orcid.org/0000-0003-4467-4915</orcidid><orcidid>https://orcid.org/0000-0003-0655-0479</orcidid><orcidid>https://orcid.org/0000-0001-7198-4199</orcidid></search><sort><creationdate>20220406</creationdate><title>Distributed agent-based deep reinforcement learning for large scale traffic signal control</title><author>Wu, Qiang ; Wu, Jianqing ; Shen, Jun ; Du, Bo ; Telikani, Akbar ; Fahmideh, Mahdi ; Liang, Chao</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c264t-ff0738ac360ae09c6fb7c3185170ec9040ccacb0dc6e788352945150f80127753</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Algorithms</topic><topic>Computer architecture</topic><topic>Computer networks</topic><topic>Deep learning</topic><topic>Distributed computing architecture</topic><topic>Distributed processing</topic><topic>Game theory</topic><topic>Internet of Things</topic><topic>Machine learning</topic><topic>Nash Equilibrium</topic><topic>Nash-A3C</topic><topic>Reinforcement learning</topic><topic>Traffic congestion</topic><topic>Traffic control</topic><topic>Traffic engineering</topic><topic>Traffic signal control</topic><topic>Traffic signals</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Wu, Qiang</creatorcontrib><creatorcontrib>Wu, Jianqing</creatorcontrib><creatorcontrib>Shen, Jun</creatorcontrib><creatorcontrib>Du, Bo</creatorcontrib><creatorcontrib>Telikani, Akbar</creatorcontrib><creatorcontrib>Fahmideh, Mahdi</creatorcontrib><creatorcontrib>Liang, Chao</creatorcontrib><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Technology Research Database</collection><collection>Library & Information Sciences Abstracts (LISA)</collection><collection>Library & Information Science Abstracts (LISA)</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>Knowledge-based systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Wu, Qiang</au><au>Wu, Jianqing</au><au>Shen, Jun</au><au>Du, Bo</au><au>Telikani, Akbar</au><au>Fahmideh, Mahdi</au><au>Liang, Chao</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Distributed agent-based deep reinforcement learning for large scale traffic signal control</atitle><jtitle>Knowledge-based systems</jtitle><date>2022-04-06</date><risdate>2022</risdate><volume>241</volume><spage>108304</spage><pages>108304-</pages><artnum>108304</artnum><issn>0950-7051</issn><eissn>1872-7409</eissn><abstract>Traffic signal control (TSC) is an established yet challenging engineering solution that alleviates traffic congestion by coordinating vehicles’ movements at road intersections. Theoretically, reinforcement learning (RL) is a promising method for adaptive TSC in complex urban traffic networks. However, current TSC systems still rely heavily on simplified rule-based methods in practice. In this paper, we propose: (1) two game theory-aided RL algorithms leveraging Nash Equilibrium and RL, namely Nash Advantage Actor–Critic (Nash-A2C) and Nash Asynchronous Advantage Actor–Critic (Nash-A3C); (2) a distributed computing Internet of Things (IoT) architecture for traffic simulation, which is more suitable for distributed TSC methods like the Nash-A3C deployment in its fog layer. We apply both methods in our computing architecture and obtain better performance than benchmark TSC methods by 22.1% and 9.7% reduction of congestion time and network delay, respectively.</abstract><cop>Amsterdam</cop><pub>Elsevier B.V</pub><doi>10.1016/j.knosys.2022.108304</doi><orcidid>https://orcid.org/0000-0002-9403-7140</orcidid><orcidid>https://orcid.org/0000-0003-4467-4915</orcidid><orcidid>https://orcid.org/0000-0003-0655-0479</orcidid><orcidid>https://orcid.org/0000-0001-7198-4199</orcidid></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0950-7051 |
| ispartof | Knowledge-based systems, 2022-04, Vol.241, p.108304, Article 108304 |
| issn | 0950-7051 1872-7409 |
| language | eng |
| recordid | cdi_proquest_journals_2642939652 |
| source | Access via ScienceDirect (Elsevier) |
| subjects | Algorithms Computer architecture Computer networks Deep learning Distributed computing architecture Distributed processing Game theory Internet of Things Machine learning Nash Equilibrium Nash-A3C Reinforcement learning Traffic congestion Traffic control Traffic engineering Traffic signal control Traffic signals |
| title | Distributed agent-based deep reinforcement learning for large scale traffic signal control |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-16T01%3A32%3A30IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Distributed%20agent-based%20deep%20reinforcement%20learning%20for%20large%20scale%20traffic%20signal%20control&rft.jtitle=Knowledge-based%20systems&rft.au=Wu,%20Qiang&rft.date=2022-04-06&rft.volume=241&rft.spage=108304&rft.pages=108304-&rft.artnum=108304&rft.issn=0950-7051&rft.eissn=1872-7409&rft_id=info:doi/10.1016/j.knosys.2022.108304&rft_dat=%3Cproquest_cross%3E2642939652%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2642939652&rft_id=info:pmid/&rft_els_id=S095070512200106X&rfr_iscdi=true |