Decentralized and Communication-Based Multi-Agent Traffic Signal Control Model Employing a Graph Representation for the State

Reinforcement learning (RL) has emerged as an effective approach for signal control, and several studies attempt to apply RL to network-wide signal control problems. Most existing methods for network-wide signal controls simply apply the pretrained model independently to the network or simply expand the observation range for the neighbors. While several recent studies have proposed cooperative signal control models using multi-agent reinforcement learning (MARL) approaches, they have not considered the transferability of the policy. This issue becomes more critical in treating a multi-agent signal control problem, since there exist innumerable possibilities of data distributions for the demand patterns. Hence, this research aims to develop a transferable and cooperative multi-agent traffic signal control model. As the key idea, we propose a decentralized and communication-based MARL approach that considers the spatial correlation of traffic dynamics in the model design and employs a graph representation for the state. Three experiments in simulated environments of the Gangnam district in Seoul, South Korea are conducted to 1) validate the model, 2) assess the policy's transferability, and 3) evaluate the efficiency of the multi-agent cooperation. The experimental results show that the proposed model obtains a transferable policy so that it adapts to the unexperienced demand scenario. In addition, it is concluded that the joint action of the proposed RL model cooperatively rebalances the traffic demands so that it improves the efficiency of the network-wide signal controls.