Decision making of autonomous vehicles in lane change scenarios: Deep reinforcement learning approaches with risk awareness

被引:164
作者
Li, Guofa [1 ]
Yang, Yifan [1 ]
Li, Shen [2 ]
Qu, Xingda [1 ]
Lyu, Nengchao [3 ]
Li, Shengbo Eben [4 ]
机构
[1] Shenzhen Univ, Inst Human Factors & Ergon, Coll Mechatron & Control Engn, Shenzhen 518060, Peoples R China
[2] Tsinghua Univ, Dept Civil Engn, Beijing 100084, Peoples R China
[3] Wuhan Univ Technol, Intelligent Transportat Syst Res Ctr, Wuhan 430063, Peoples R China
[4] Tsinghua Univ, Sch Vehicle & Mobil, State Key Lab Automot Safety & Energy, Beijing 100084, Peoples R China
基金
中国国家自然科学基金;
关键词
Driving safety; Driving risk; Autonomous vehicle; Driver assistance system; Reinforcement learning; FRAMEWORK; EFFICIENT; BEHAVIOR;
D O I
10.1016/j.trc.2021.103452
中图分类号
U [交通运输];
学科分类号
08 ; 0823 ;
摘要
Driving safety is the most important element that needs to be considered for autonomous vehicles (AVs). To ensure driving safety, we proposed a lane change decision-making framework based on deep reinforcement learning to find a risk-aware driving decision strategy with the minimum expected risk for autonomous driving. Firstly, a probabilistic-model based risk assessment method was proposed to assess the driving risk using position uncertainty and distance-based safety metrics. Then, a risk aware decision making algorithm was proposed to find a strategy with the minimum expected risk using deep reinforcement learning. Finally, our proposed methods were evaluated in CARLA in two scenarios (one with static obstacles and one with dynamically moving vehicles). The results show that our proposed methods can generate robust safe driving strategies and achieve better driving performances than previous methods.
引用
收藏
页数:18
相关论文
共 64 条
[41]   Human-level control through deep reinforcement learning [J].
Mnih, Volodymyr ;
Kavukcuoglu, Koray ;
Silver, David ;
Rusu, Andrei A. ;
Veness, Joel ;
Bellemare, Marc G. ;
Graves, Alex ;
Riedmiller, Martin ;
Fidjeland, Andreas K. ;
Ostrovski, Georg ;
Petersen, Stig ;
Beattie, Charles ;
Sadik, Amir ;
Antonoglou, Ioannis ;
King, Helen ;
Kumaran, Dharshan ;
Wierstra, Daan ;
Legg, Shane ;
Hassabis, Demis .
NATURE, 2015, 518 (7540) :529-533
[42]  
Mokhtari K, 2021, ARXIV PREPRINT ARXIV
[43]  
National Highway Traffic Safety Administration (NHTSA), 2019, BRANDWONDENRAPPORTAG
[44]   Decision-Making Framework for Autonomous Driving at Road Intersections: Safeguarding Against Collision, Overly Conservative Behavior, and Violation Vehicles [J].
Noh, Samyeul .
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, 2019, 66 (04) :3275-3286
[45]   Decision-Making Framework for Automated Driving in Highway Environments [J].
Noh, Samyeul ;
An, Kyounghwan .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2018, 19 (01) :58-71
[46]   A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles [J].
Paden, Brian ;
Cap, Michal ;
Yong, Sze Zheng ;
Yershov, Dmitry ;
Frazzoli, Emilio .
IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, 2016, 1 (01) :33-55
[47]  
Pan X., 2017, ARXIV170403952
[48]   Deep reinforcement learning enabled self-learning control for energy efficient driving [J].
Qi, Xuewei ;
Luo, Yadan ;
Wu, Guoyuan ;
Boriboonsomsin, Kanok ;
Barth, Matthew .
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES, 2019, 99 :67-81
[49]   Driving in Dense Traffic with Model-Free Reinforcement Learning [J].
Saxena, Dhruv Mauria ;
Bae, Sangjae ;
Nakhaei, Alireza ;
Fujimura, Kikuo ;
Likhachev, Maxim .
2020 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2020, :5385-5392
[50]   Human-Centered Risk Assessment of an Automated Vehicle Using Vehicular Wireless Communication [J].
Shin, Donghoon ;
Kim, Beomjun ;
Yi, Kyongsu ;
Carvalho, Ashwin ;
Borrelli, Francesco .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2019, 20 (02) :667-681