Reinforcement Learning Based Multi-objective Eco-driving Strategy in Urban Scenarios

To improve the ride experience of connected and automated vehicle in complex urban traffic scenarios,this paper proposes a deep reinforcement learning based multi-objective eco-driving strategy that considers driving safety,energy economy,ride comfort,and travel efficiency. Firstly,the state space,action space,and multi-objective reward function of the eco-driving strategy are constructed based on the Markov decision process. Secondly,the car-following safety model and traffic light safety model are designed to provide safety speed suggestion for the eco-driving strategy. Thirdly,the composite multi-objective reward function design method that integrates safety constraints and shaping functions is proposed to ensure training convergence and optimization performance of the deep reinforcement learning agent. Finally,the effectiveness of the proposed method is verified through hardware-in-the-loop experiments. The results show that the proposed strategy can be applied in real-time on the onboard vehicle control unit. Compared to the eco-driving strategy based on the intelligent driver model,the proposed strategy improves energy economy,ride comfort,and travel efficiency of the vehicle while satisfying the driving safety constraints. © 2023 SAE-China. All rights reserved.