Deep reinforcement learning-based energy management system enhancement using digital twin for electric vehicles

Compared to conventional engine-based powertrains, electrified powertrain exhibit increased energy efficiency and reduced emissions, making electrification a key goal for the automotive industry. For a vehicle with a hybrid energy storage system, its performance and lifespan are substantially affected by the energy management system. Reinforcement learning-based methods are gaining popularity in vehicle energy management, but most of the literature in this area focuses on pure simulation, while hardware implementation is still limited. This paper introduces the digital twin methodology to enhance the reinforcement learning-based energy management system for battery and ultracapacitor electric vehicles. The digital twin model can exploit the bilateral interdependency between the virtual model and the actual system, which improves the control performance of the energy management system in real-time control. The physical model is established based on a hardware-in-theloop simulation platform. In addition, battery degradation is also considered for prolonging the battery lifespan to reduce operating costs. The validation results of the trained reinforcement learning agent illustrate that the digital twin-enhanced Q-learning energy management system improves the energy efficiency by 7.08 % and reduces the battery degradation by 25.28 %.