Enhancing Cooperation of Vehicle Merging Control in Heavy Traffic Using Communication-Based Soft Actor-Critic Algorithm

被引:15
作者
Li, Meng [1 ]
Li, Zhibin [1 ]
Wang, Shunchao [1 ]
Zheng, Si [1 ]
机构
[1] Southeast Univ, Sch Transportat, Nanjing 210096, Peoples R China
基金
中国国家自然科学基金;
关键词
Merging strategy; soft actor-critic; mixed heavy traffic; communication protocol; VARIABLE-SPEED LIMIT; AUTONOMOUS VEHICLES; AUTOMATED VEHICLES; RAMP; STRATEGY; MODEL;
D O I
10.1109/TITS.2022.3221450
中图分类号
TU [建筑科学];
学科分类号
0813 ;
摘要
A promising way to improve efficiency in highway on-ramp regions is to control connected and automated vehicles (CAVs) to pass the merging section sequentially. The primary objective of this paper is to incorporate the reinforcement learning (RL) technique into vehicle merging control to achieve global cooperation. A communication protocol among RL agents is integrated with the Soft Actor-Critic (CSAC) algorithm. The parallel SAC agents with a parameter-sharing structure cooperate to optimize the common reward function. It enables CAVs to know the actions of each other so that they proactively negotiate to adjust speeds. We designed a parsimonious state representation containing crucial merging information to speed up the RL training. The effects of the merging control strategy were investigated with the simulation model in scenarios with different CAV penetration rates and traffic flow compositions. Results showed that the CSAC-based merging strategy generated collision-free merging trajectories with a short travel time while guaranteeing traffic safety in various traffic conditions. Four baselines (including two rule-based and two RL-based merging strategies) were applied in the same scenarios for comparison. It was found that the proposed merging strategy took the safest merging behaviors (zero traffic conflicts) and reduced the most significant travel time (56.9%).
引用
收藏
页码:6491 / 6506
页数:16
相关论文
共 51 条
[1]   Considering various ALINEA ramp metering strategies for crash risk mitigation on freeways under congested regime [J].
Abdel-Aty, Mohamed ;
Dhindsa, Albinder ;
Gayah, Vikash .
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES, 2007, 15 (02) :113-134
[2]  
Bai F, 2010, MOBICOM 10 & MOBIHOC 10: PROCEEDINGS OF THE 16TH ANNUAL INTERNATIONAL CONFERENCE ON MOBILE COMPUTING AND NETWORKING AND THE 11TH ACM INTERNATIONAL SYMPOSIUM ON MOBILE AD HOC NETWORKING AND COMPUTING, P329
[3]   Model predictive control for ramp metering of motorway traffic: A case study [J].
Bellemans, T. ;
De Schutter, B. ;
De Moor, B. .
CONTROL ENGINEERING PRACTICE, 2006, 14 (07) :757-767
[4]  
Bouton M, 2019, IEEE INT C INTELL TR, P3441, DOI [10.1109/ITSC.2019.8916924, 10.1109/itsc.2019.8916924]
[5]   Key feature selection and risk prediction for lane-changing behaviors based on vehicles' trajectory data [J].
Chen, Tianyi ;
Shi, Xiupeng ;
Wong, Yiik Diew .
ACCIDENT ANALYSIS AND PREVENTION, 2019, 129 (156-169) :156-169
[6]  
Chen XB, 2007, GLOB TELECOMM CONF, P151
[7]  
Chen Z. Li, 2021, ARXIV
[8]   Effect of cooperative merging on the synchronous flow phase of traffic [J].
Davis, LC .
PHYSICA A-STATISTICAL MECHANICS AND ITS APPLICATIONS, 2006, 361 (02) :606-618
[9]   A Rule-Based Cooperative Merging Strategy for Connected and Automated Vehicles [J].
Ding, Jishiyu ;
Li, Li ;
Peng, Huei ;
Zhang, Yi .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2020, 21 (08) :3436-3446
[10]   An Introduction to Deep Reinforcement Learning [J].
Francois-Lavet, Vincent ;
Henderson, Peter ;
Islam, Riashat ;
Bellemare, Marc G. ;
Pineau, Joelle .
FOUNDATIONS AND TRENDS IN MACHINE LEARNING, 2018, 11 (3-4) :219-354