Day-ahead scheduling based on reinforcement learning with hybrid action space

被引:1
作者
Cao Jingyu [1 ]
Dong Lu [2 ]
Sun Changyin [1 ]
机构
[1] Southeast Univ, Sch Automat, Nanjing 210096, Peoples R China
[2] Southeast Univ, Sch Cyber Sci & Engn, Nanjing 211189, Peoples R China
基金
中国国家自然科学基金;
关键词
day-ahead scheduling; active distribution network (ADN); reinforcement learning; hybrid action space; DEMAND-SIDE MANAGEMENT; OPTIMIZATION; HOME; MINIMIZATION; GENERATION;
D O I
10.23919/JSEE.2022.000064
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Driven by the improvement of the smart grid, the active distribution network (ADN) has attracted much attention due to its characteristic of active management. By making full use of electricity price signals for optimal scheduling, the total cost of the ADN can be reduced. However, the optimal dayahead scheduling problem is challenging since the future electricity price is unknown. Moreover, in ADN, some schedulable variables are continuous while some schedulable variables are discrete, which increases the difficulty of determining the optimal scheduling scheme. In this paper, the day-ahead scheduling problem of the ADN is formulated as a Markov decision process (MDP) with continuous-discrete hybrid action space. Then, an algorithm based on multi-agent hybrid reinforcement learning (HRL) is proposed to obtain the optimal scheduling scheme. The proposed algorithm adopts the structure of centralized training and decentralized execution, and different methods are applied to determine the selection policy of continuous scheduling variables and discrete scheduling variables. The simulation experiment results demonstrate the effectiveness of the algorithm.
引用
收藏
页码:693 / 705
页数:13
相关论文
共 39 条
[1]   Demand side management in a smart micro-grid in the presence of renewable generation and demand response [J].
Aghajani, G. R. ;
Shayanfar, H. A. ;
Shayeghi, H. .
ENERGY, 2017, 126 :622-637
[2]   Demand Response Strategy Based on Reinforcement Learning and Fuzzy Reasoning for Home Energy Management [J].
Alfaverh, Fayiz ;
Denai, M. ;
Sun, Yichuang .
IEEE ACCESS, 2020, 8 :39310-39321
[3]   DYNAMIC PROGRAMMING [J].
BELLMAN, R .
SCIENCE, 1966, 153 (3731) :34-&
[4]  
BESTER C J, MULTIPASS Q NETWORKS
[5]   Load Scheduling for an Electric Water Heater With Forecasted Price Using Deep Reinforcement Learning [J].
Cao, Jingyu ;
Dong, Lu ;
Xue, Lei .
2020 CHINESE AUTOMATION CONGRESS (CAC 2020), 2020, :2500-2505
[6]   Distributed Deep Reinforcement Learning for Intelligent Load Scheduling in Residential Smart Grids [J].
Chung, Hwei-Ming ;
Maharjan, Sabita ;
Zhang, Yan ;
Eliassen, Frank .
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, 2021, 17 (04) :2752-2763
[7]   A Demand Response Scheme in Smart Grid with Clustering of Residential Customers [J].
Dai, Bijian ;
Wang, Ran ;
Zhu, Kun ;
Hao, Jie ;
Wang, Ping .
2019 IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS, CONTROL, AND COMPUTING TECHNOLOGIES FOR SMART GRIDS (SMARTGRIDCOMM), 2019,
[8]   The optimization of demand response programs in smart grids [J].
Derakhshan, Ghasem ;
Shayanfar, Heidar Ali ;
Kazemi, Ahad .
ENERGY POLICY, 2016, 94 :295-306
[9]   Mean-Variance Optimization-Based Energy Storage Scheduling Considering Day-Ahead and Real-Time LMP Uncertainties [J].
Fang, Xin ;
Hodge, Bri-Mathias ;
Bai, Linquan ;
Cui, Hantao ;
Li, Fangxing .
IEEE TRANSACTIONS ON POWER SYSTEMS, 2018, 33 (06) :7292-7295
[10]  
FU H T, DEEP MULTIAGENT REIN