An AGC Dynamic Optimization Method Based on Proximal Policy Optimization

被引:4
作者
Liu, Zhao [1 ]
Li, Jiateng [2 ]
Zhang, Pei [1 ]
Ding, Zhenhuan [3 ]
Zhao, Yanshun [1 ]
机构
[1] Beijing Jiaotong Univ, Sch Elect Engn, Beijing, Peoples R China
[2] China Elect Power Res Inst, Dept Artificial Intelligence Applicat, Beijing, Peoples R China
[3] Anhui Univ, Sch Artificial Intelligence, Hefei, Peoples R China
基金
中国国家自然科学基金;
关键词
automatic generation control; advanced optimization strategy; deep reinforcement learning; renewable energy; proximal policy optimization; AUTOMATIC-GENERATION CONTROL; LOAD-FREQUENCY CONTROL; REINFORCEMENT LEARNING-METHOD; POWER; ALGORITHM; DESIGN; SYSTEM; CONTROLLER; MANAGEMENT;
D O I
10.3389/fenrg.2022.947532
中图分类号
TE [石油、天然气工业]; TK [能源与动力工程];
学科分类号
0807 ; 0820 ;
摘要
The increasing penetration of renewable energy introduces more uncertainties and creates more fluctuations in power systems than ever before, which brings great challenges for automatic generation control (AGC). It is necessary for grid operators to develop an advanced AGC strategy to handle fluctuations and uncertainties. AGC dynamic optimization is a sequential decision problem that can be formulated as a discrete-time Markov decision process. Therefore, this article proposes a novel framework based on proximal policy optimization (PPO) reinforcement learning algorithm to optimize power regulation among each AGC generator in advance. Then, the detailed modeling process of reward functions and state and action space designing is presented. The application of the proposed PPO-based AGC dynamic optimization framework is simulated on a modified IEEE 39-bus system and compared with the classical proportional-integral (PI) control strategy and other reinforcement learning algorithms. The results of the case study show that the framework proposed in this article can make the frequency characteristic better satisfy the control performance standard (CPS) under the scenario of large fluctuations in power systems.
引用
收藏
页数:13
相关论文
共 40 条
[1]   Optimal AGC tuning with genetic algorithms [J].
AbdelMagid, YL ;
Dawoud, MM .
ELECTRIC POWER SYSTEMS RESEARCH, 1996, 38 (03) :231-238
[2]  
Atic N., 2003, POWER ENG SOC GEN M
[3]   Impacts of wind power minute-to-minute variations on power system operation [J].
Banakar, Hadi ;
Luo, Changling ;
Ooi, Boon Teck .
IEEE TRANSACTIONS ON POWER SYSTEMS, 2008, 23 (01) :150-160
[4]   APPLICATION OF NEURAL NETWORKS TO LOAD-FREQUENCY CONTROL IN POWER-SYSTEMS [J].
BEAUFAYS, F ;
ABDELMAGID, Y ;
WIDROW, B .
NEURAL NETWORKS, 1994, 7 (01) :183-194
[5]   OPTIMUM LOAD-FREQUENCY SAMPLED-DATA CONTROL WITH RANDOMLY VARYING SYSTEM DISTURBANCES [J].
BOHN, EV ;
MINIESY, SM .
IEEE TRANSACTIONS ON POWER APPARATUS AND SYSTEMS, 1972, 91 (05) :1916-&
[6]   Load frequency control using genetic-algorithm based fuzzy gain scheduling of PI controllers [J].
Chang, CS ;
Fu, WH ;
Wen, FS .
ELECTRIC MACHINES AND POWER SYSTEMS, 1998, 26 (01) :39-52
[7]  
Concordia C, 1953, Trans. Am. Inst. Electr. Eng. III: Power Appar. Syst., V72, P562
[8]   Automatic generation control using disrupted oppositional based gravitational search algorithm optimised sliding mode controller under deregulated environment [J].
Dahiya, Preeti ;
Sharma, Veena ;
Naresh, Ram .
IET GENERATION TRANSMISSION & DISTRIBUTION, 2016, 10 (16) :3995-4005
[9]   Deep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid Operations [J].
Duan, Jiajun ;
Shi, Di ;
Diao, Ruisheng ;
Li, Haifeng ;
Wang, Zhiwei ;
Zhang, Bei ;
Bian, Desong ;
Yi, Zhehan .
IEEE TRANSACTIONS ON POWER SYSTEMS, 2020, 35 (01) :814-817
[10]  
Elgerd O.I., 2007, IEEE T POWER APPARAT, V89, P556