Learning Automated Driving in Complex Intersection Scenarios Based on Camera Sensors: A Deep Reinforcement Learning Approach

Making proper decisions at intersections that are one of the most dangerous and sophisticated driving scenarios is full of challenges, especially for autonomous vehicles (AVs). The existing decision-making approaches for AVs at intersections are limited as they only consider driving safety in simple intersection scenarios while sacrificing travel efficiency and driving comfort. To solve this issue, a decision-making structure motivated by deep reinforcement learning was proposed for autonomous driving at complex intersection scenarios based on long short-term memory (LSTM). The mapping relationship between traffic images collected from camera sensors and AVs' actions was established by constructing convolutional-recurrent neural networks in a decision-making framework. Traffic images collected from camera sensors at two different timesteps were used to understand the relative motion information between AVs and other vehicles. To model the interaction between the AV and other vehicles, Markov decision process was used. The deep Q-network (DQN) algorithm was applied to generate the optimal driving policy that could comprehensively consider driving safety, travel efficiency and driving comfort. Three crash-prone complex intersection scenarios were reconstructed in CARLA (car learning to act) to evaluate the performance of our proposed method. The results indicate that our method can make AV drive through intersections safely and efficiently with desirable driving comfort in all the examined scenarios.