Optimal Control for Interconnected Multi-Area Power Systems With Unknown Dynamics: An Off-Policy Q-Learning Method

This brief focuses on the optimal control problem for interconnected multi-area power systems with unknown dynamics. Firstly, in this brief, singular phenomena are considered to describe interconnected multi-area power systems. Then, the optimal controller is solved using a model-based control method via Bellman ' s optimality principle, which requires the system dynamics. In this situation, a model-free off-policy Q-learning method is proposed considering the case where the system dynamics are completely unknown. Compared with on-policy Q-learning, the off-policy Q-learning method enhances data exploration capabilities. During the learning process, the off-policy Q-learning method uses the online information of state and input to iteratively solve the game algebraic Riccati equation without requiring system dynamics, which is more practical. Furthermore, the optimality of the proposed method is demonstrated. Finally, a simulation example is used to demonstrate the effectiveness of the designed method.