Long-term deep reinforcement learning for real-time economic generation control of cloud energy storage systems with varying structures

Energy storage systems play a crucial role in modern power systems. Consequently, a mixed cloud energy storage (CES) system is proposed. The mixed CES system comprises consumers and prosumers. The consumers can only consume energy. The prosumers can either produce or consume energy at different time intervals. The proposed mixed CES system is designed for investigating the generation control challenges in mixed interconnected power systems. To optimize the active power balance and economic efficiency of the mixed CES system, a long-term deep reinforcement learning (LDRL) artificial intelligence approach is proposed as the real-time economic generation controller to control the mixed CES system. The LDRL consists of a reinforcement mechanism and two models: a long short-term memory model for economic dispatch and a deep neural networks model for smart generation control. The reinforcement framework updates policies for the prosumers. The efficacy of proposed method is validated across three mixed systems, i.e., the improved IEEE 300-bus, Polish 2383-bus, and mixed systems with varying structures. The numerical simulations verify that the LDRL method can efficiently and economically control the mixed CES systems with diverse structures.