Stochastic Optimal Relaxed Automatic Generation Control in Non-Markov Environment Based on Multi-Step Q(λ) Learning

被引:86
作者
Yu, Tao
Zhou, Bin [1 ]
Chan, Ka Wing [1 ]
Chen, Liang [2 ]
Yang, Bo
机构
[1] Hong Kong Polytech Univ, Dept Elect Engn, Hong Kong, Hong Kong, Peoples R China
[2] China So Power Grid Co, Guangdong Power Dispatch & Commun Ctr, Guangzhou 510600, Guangdong, Peoples R China
基金
中国国家自然科学基金;
关键词
AGC; CPS; multi-step Q(lambda) learning; non-Markov environment; relaxed control; stochastic optimization; FREQUENCY CONTROL; NERCS;
D O I
10.1109/TPWRS.2010.2102372
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
This paper proposes a stochastic optimal relaxed control methodology based on reinforcement learning (RL) for solving the automatic generation control (AGC) under NERC's control performance standards (CPS). The multi-step Q(lambda) learning algorithm is introduced to effectively tackle the long time-delay control loop for AGC thermal plants in non-Markov environment. The moving averages of CPS1/ACE are adopted as the state feedback input, and the CPS control and relaxed control objectives are formulated as multi-criteria reward function via linear weighted aggregate method. This optimal AGC strategy provides a customized platform for interactive self-learning rules to maximize the long-run discounted reward. Statistical experiments show that the RL theory based Q(lambda) controllers can effectively enhance the robustness and dynamic performance of AGC systems, and reduce the number of pulses and pulse reversals while the CPS compliances are ensured. The novel AGC scheme also provides a convenient way of controlling the degree of CPS compliance and relaxation by online tuning relaxation factors to implement the desirable relaxed control.
引用
收藏
页码:1272 / 1282
页数:11
相关论文
共 31 条
[1]  
Ahamed TPI, 2002, ELECTR POW SYST RES, V63, P9, DOI 10.1016/S0378-7796(02)00088-3
[2]  
[Anonymous], OP MOD CHIN SO POW G
[3]   Technical and economic aspects of using a long submarine HVDC connection for frequency control [J].
Bakken, BH ;
Faanes, HH .
IEEE TRANSACTIONS ON POWER SYSTEMS, 1997, 12 (03) :1252-1257
[4]  
Bhattacharyya SP, 2009, AUTOM CONTROL ENG SE, V33, pXVII
[5]   A methodology for the redesign of frequency control for AC networks [J].
Chown, GA ;
Wigdorowitz, B .
IEEE TRANSACTIONS ON POWER SYSTEMS, 2004, 19 (03) :1546-1554
[6]  
Elgerd OI., 1982, ELECT ENERGY SYSTEMS
[7]   Power systems stability control: Reinforcement learning framework [J].
Ernst, D ;
Glavic, M ;
Wehenkel, L .
IEEE TRANSACTIONS ON POWER SYSTEMS, 2004, 19 (01) :427-435
[8]   NERC compliant load frequency control design using fuzzy rules [J].
Feliachi, A ;
Rerkpreedapong, D .
ELECTRIC POWER SYSTEMS RESEARCH, 2005, 73 (02) :101-106
[9]  
Gao Zonghe, 2004, Automation of Electric Power Systems, V28, P78
[10]  
Garrity TF, 2008, IEEE POWER ENERGY M, V6, P38, DOI [10.1109/MPE.2007.915181, 10.1109/MPE.20G7.915181]