A Multi-Step Unified Reinforcement Learning Method for Automatic Generation Control in Multi-Area Interconnected Power Grid

As the penetration rate of new energies, energy storage devices and electric vehicles increases continuously in power grid, power grid faces strong random disturbances, as well as the issues of downfall in frequency control ability induced by the factors such as system inertia reduction and frequency control under-capacity in traditional power units. Therefore, an algorithm of automatic generation control with DQ(sigma, lambda) is proposed for the multi-area interconnected power grid. By integrating the eligibility trace, DQ(sigma, lambda) adopts the principle of double Q-learning to unify double Q(lambda) and double Expected-Sarsa(lambda), thereby the over-estimation of Q in the algorithm and the phenomenon of over-fitting can be avoided. The simulations are provided to demonstrate the effectiveness of the proposed algorithm, where the improved IEEE standard two-area load-frequency control model and the model of the multi-area interconnected power grid based on Central China Power Grid are adopted. The results show that the phenomena of random disturbances and frequency instability can be eliminated in multi-area interconnected power grid. The convergence is faster and the dynamic performance is better in the proposed algorithm compared with the traditional algorithms.