DiffusionLight:a multi-agent reinforcement learning approach for traffic signal control based on shortcut-diffusion model

Numerous researches have shown that the Reinforcement Learning(RL) is an effective solution to solve large-scale traffic signal control(TSC) problems. However, facing multi-scenario and emergencies, cooperative control of traffic signals at multi-intersection becomes a challenging multi-agent reinforcement learning(MARL) process. In order to solve real-world problems, this paper proposes a MARL algorithm called DiffusionLight, which combines Shortcut-Diffusion Model(SDM) and Soft Actor-Critic(SAC), and a fast Diffusion model to solve the traffic signal cooperative control problem of multi-scenario and multi-intersection. DiffusionLight has the stable characteristics and powerful expression ability of the SDM as the strategy network to solve the action space, while SAC is used as the value network to better explore the solution space. Experimental results show that DiffusionLight exhibits better stability compared to the baseline algorithm in the face of multi-scenario TSC and burst data anomalies, and as well as excellent performance on multiple public datasets of grid and arterial traffic networks.