Physical model-assisted deep reinforcement learning for energy management optimization of industrial electric-hydrogen coupling system with hybrid energy storage

Utilizing renewable energy sources (RESs), such as wind and solar, to convert electrical energy into hydrogen energy for industrial users with different types of energy storage can enhance the integration of green electricity, thereby replacing fossil fuels like natural gas. In the energy management optimization of industrial electric-hydrogen coupling system (EHCS) with high RESs integration, the traditional optimization methods cannot effectively address the nonlinear characteristics of equipment and stochastic RES generation, while the modelfree reinforcement learning methods struggle to improve the training efficiency and may lead to constraint violation. To address this challenge, this paper proposes a short time scale energy management approach for EHCS based on physical models assisted deep reinforcement learning (DRL). Firstly, the energy management optimization model of the EHCS is developed based on the operation characteristics of the equipment within the system. Secondly, the energy management model of EHCS is formulated as a DRL framework. The DRL agent is efficiently trained with the assistance of the action transformation which takes into account the physical characteristics of the equipment during operation. Finally, a case study is conducted to verify the effectiveness of the proposed model for achieving high-efficiency agent training, reducing decision-making time for equipment scheduling, addressing the stochastic RES generation, and realizing the economic operation of EHCS with good performance based on the equipment characteristics.