A Novel Self-tuning CPS Controller Based on Q-learning Method

This paper describes an application of Q-learning method based on-line self-tuning control methodology to solve the automatic generation control (AGC) under NERC's new control performance standards (CPS). The AGC problem is a stochastic multistage decision problem, which can be modeled as a Markov Decision Process (MDP). This model-free Q-learning algorithm regards, the CPS values as the rewards from the interconnected power systems. By regulating a closed-loop CPS control rule to maximize the total reward in the procedure of on-line learning, the optimal CPS control strategy can gradually obtained. The case study shows that after adding the Q-learning controller, the robustness and adaptability of AGC system is enhanced obviously and the CPS compliance is ensured.