A Coordination Graph Based Framework for Network Traffic Signal Control

The efficiency of road networks affects the daily activities of each stakeholder. Multi-agent reinforcement learning (MARL) has emerged as a method for managing network traffic signal control (TSC). It treats each intersection as an agent and coordinates their actions to enhance overall performance. A critical issue is enabling agents to appropriately and systematically respond to network demand changes. In response, this study proposes a coordination graph-based framework. It considers two adjacent intersections as a pair and updates coordination graphs periodically based on observed demand patterns, determining which intersection pairs should be coordinated. Within this framework, an adaptive TSC method based on reinforcement learning is designed for isolated intersections. Furthermore, paired intersections are jointly controlled using a modified max-plus algorithm. The coordination graph is solved considering factors such as traffic demand and intersection spacing, employing a decomposition method named "snake game solver". Experimental results show that the individual learning scheme resulted in robust control and quick adaptability to traffic fluctuations. However, the coordination learning scheme only led to improvements when the inter-demand between intersections was sufficiently high and the spacing was short. The numerical study suggests that this control framework could enhance network efficiency compared to other MARL-TSC methods.