Driver-like decision-making method for vehicle longitudinal autonomous driving based on deep reinforcement learning

被引:8
作者
Gao, Zhenhai [1 ]
Yan, Xiangtong [1 ]
Gao, Fei [1 ]
He, Lei [1 ]
机构
[1] Jilin Univ, State Key Lab Automot Simulat & Control, 5988 Renmin St, Changchun 130022, Jilin, Peoples R China
基金
中国国家自然科学基金; 国家重点研发计划;
关键词
Autonomous driving; decision-making algorithm; deep reinforcement learning; Deep Q-Network; deep deterministic policy gradient;
D O I
10.1177/09544070211063081
中图分类号
TH [机械、仪表工业];
学科分类号
0802 ;
摘要
Decision-making is one of the key parts of the research on vehicle longitudinal autonomous driving. Considering the behavior of human drivers when designing autonomous driving decision-making strategies is a current research hotspot. In longitudinal autonomous driving decision-making strategies, traditional rule-based decision-making strategies are difficult to apply to complex scenarios. Current decision-making methods that use reinforcement learning and deep reinforcement learning construct reward functions designed with safety, comfort, and economy. Compared with human drivers, the obtained decision strategies still have big gaps. Focusing on the above problems, this paper uses the driver's behavior data to design the reward function of the deep reinforcement learning algorithm through BP neural network fitting, and uses the deep reinforcement learning DQN algorithm and the DDPG algorithm to establish two driver-like longitudinal autonomous driving decision-making models. The simulation experiment compares the decision-making effect of the two models with the driver curve. The results shows that the two algorithms can realize driver-like decision-making, and the consistency of the DDPG algorithm and human driver behavior is higher than that of the DQN algorithm, the effect of the DDPG algorithm is better than the DQN algorithm.
引用
收藏
页码:3060 / 3070
页数:11
相关论文
共 28 条
[1]  
Chae H, 2017, IEEE INT C INTELL TR
[2]   Cooperative Adaptive Cruise Control: A Reinforcement Learning Approach [J].
Desjardins, Charles ;
Chaib-draa, Brahim .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2011, 12 (04) :1248-1260
[3]   Decision-making method for vehicle longitudinal automatic driving based on reinforcement Q-learning [J].
Gao, Zhenhai ;
Sun, Tianjun ;
Xiao, Hongwei .
INTERNATIONAL JOURNAL OF ADVANCED ROBOTIC SYSTEMS, 2019, 16 (03)
[4]   Control mode switching strategy for ACC based on intuitionistic fuzzy set multi-attribute decision making method [J].
Gao, Zhenhai ;
Wang, Jun ;
Hu, Hongyu ;
Sun, Yiteng .
JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2016, 31 (06) :2967-2974
[5]  
Girard AR, 2001, IEEE DECIS CONTR P, P1491, DOI 10.1109/CDC.2001.981105
[6]   Hybrid deep reinforcement learning based eco-driving for low-level connected and automated vehicles along signalized corridors [J].
Guo, Qiangqiang ;
Angah, Ohay ;
Liu, Zhijun ;
Ban, Xuegang .
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES, 2021, 124
[7]   End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm [J].
Hu, Hongyu ;
Lu, Ziyang ;
Wang, Qi ;
Zheng, Chengyuan .
SENSORS, 2020, 20 (18) :1-22
[8]  
Junru Yang, 2020, Proceedings of the 2020 4th CAA International Conference on Vehicular Control and Intelligence (CVCI), P656, DOI 10.1109/CVCI51460.2020.9338516
[9]  
Lillicrap T. P., 2015, ARXIV
[10]   Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Control [J].
Lin, Yuan ;
McPhee, John ;
Azad, Nasser L. .
IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, 2021, 6 (02) :221-231