DiffusionLight:a multi-agent reinforcement learning approach for traffic signal control based on shortcut-diffusion model

被引:0
作者
Yu, Jilin [1 ]
Wang, Zhiwen [1 ]
Zhang, Ruonan [1 ]
机构
[1] Guangxi Univ Sci & Technol, Coll Comp Sci & Technol, 2 Wenchang Rd, Liuzhou 545006, Guangxi, Peoples R China
基金
中国国家自然科学基金;
关键词
Diffusion model; Soft actor-critic; Shortcut theory; Multi-agent reinforcement learning; Burst data anomalies;
D O I
10.1007/s10489-025-06359-8
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Numerous researches have shown that the Reinforcement Learning(RL) is an effective solution to solve large-scale traffic signal control(TSC) problems. However, facing multi-scenario and emergencies, cooperative control of traffic signals at multi-intersection becomes a challenging multi-agent reinforcement learning(MARL) process. In order to solve real-world problems, this paper proposes a MARL algorithm called DiffusionLight, which combines Shortcut-Diffusion Model(SDM) and Soft Actor-Critic(SAC), and a fast Diffusion model to solve the traffic signal cooperative control problem of multi-scenario and multi-intersection. DiffusionLight has the stable characteristics and powerful expression ability of the SDM as the strategy network to solve the action space, while SAC is used as the value network to better explore the solution space. Experimental results show that DiffusionLight exhibits better stability compared to the baseline algorithm in the face of multi-scenario TSC and burst data anomalies, and as well as excellent performance on multiple public datasets of grid and arterial traffic networks.
引用
收藏
页数:25
相关论文
共 42 条
[1]  
Bauschke HH, 2011, CMS BOOKS MATH, P1, DOI 10.1007/978-1-4419-9467-7
[2]  
Chen C, 2020, Toward a thousand lights: decentralized deep reinforcement learning for large-scale traffic signal control, P3414
[3]  
Chen HY, 2024, Arxiv, DOI arXiv:2410.22938
[4]   Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction [J].
Chung, Hyungjin ;
Sim, Byeongsu ;
Ye, Jong Chul .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, :12403-12412
[5]  
Dey R, 2017, MIDWEST SYMP CIRCUIT, P1597, DOI 10.1109/MWSCAS.2017.8053243
[6]  
Foerster JN, 2018, AAAI CONF ARTIF INTE, P2974
[7]   Multi-agent deep reinforcement learning: a survey [J].
Gronauer, Sven ;
Diepold, Klaus .
ARTIFICIAL INTELLIGENCE REVIEW, 2022, 55 (02) :895-943
[8]  
Haarnoja T, 2018, PR MACH LEARN RES, V80
[9]  
Ho JAT, 2020, Arxiv, DOI [arXiv:2006.11239, 10.48550/arXiv.2006.11239, DOI 10.48550/ARXIV.2006.11239]
[10]   GENETIC ALGORITHMS [J].
HOLLAND, JH .
SCIENTIFIC AMERICAN, 1992, 267 (01) :66-72