Deep Reinforcement Learning-Based Demand Response for Smart Facilities Energy Management

被引:48
作者
Lu, Renzhi [1 ,2 ,3 ]
Bai, Ruichang [1 ]
Luo, Zhe [4 ]
Jiang, Junhui [5 ]
Sun, Mingyang [6 ]
Zhang, Hai-Tao [1 ,7 ]
机构
[1] Huazhong Univ Sci & Technol, Sch Artificial Intelligence & Automat, Key Lab Image Proc & Intelligent Control, Wuhan 430074, Peoples R China
[2] North China Elect Power Univ, State Key Lab Alternate Elect Power Syst Renewabl, Beijing 102206, Peoples R China
[3] Chongqing Univ Posts & Telecommun, Key Lab Ind Internet Things & Networked Control, Chongqing 400065, Peoples R China
[4] Tsinghua Univ, Tsinghua Berkeley Shenzhen Inst TBSI, Tsinghua Shenzhen Int Grad Sch, Shenzhen 518055, Peoples R China
[5] Hanyang Univ, Dept Elect Syst Engn, Ansan 15588, South Korea
[6] Zhejiang Univ, Dept Control Sci & Engn, State Key Lab Ind Control Technol, Hangzhou 310027, Peoples R China
[7] Huazhong Univ Sci & Technol, State Key Lab Digital Mfg Equipment & Technol, Wuhan 430074, Peoples R China
基金
中国国家自然科学基金;
关键词
Energy management; Load management; Feature extraction; Approximation algorithms; Predictive models; Prediction algorithms; Power system stability; Deep reinforcement learning (DRL); demand response; energy management; experimental validation; smart facility; MODEL;
D O I
10.1109/TIE.2021.3104596
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
This work proposes a novel deep reinforcement learning (DRL)-based demand response algorithm for smart facilities energy management to minimize electricity costs while maintaining a satisfaction index. Specifically, to accommodate the characteristics of the decision-making problem, long short-term memory (LSTM) units are adopted to extract discriminative features from past electricity price sequences and fed into fully connected multi-layer perceptrons (MLPs) with the measured energy and time information, then a deep Q-network is developed to approximate the optimal policy. After that, an experimental setup is constructed to investigate the effectiveness of the proposed DRL-based demand response algorithm to bridge the gap between theoretical studies and practical implementations. Numerical results demonstrate that the proposed algorithm can handle energy management well for multiple smart facilities. Moreover, the proposed algorithm outperforms the model predictive control (MPC) strategy and uncontrolled solution and is close to the theoretical optimal control method.
引用
收藏
页码:8554 / 8565
页数:12
相关论文
共 32 条
[1]   Multiagent Reinforcement Learning for Energy Management in Residential Buildings [J].
Ahrarinouri, Mehdi ;
Rastegar, Mohammad ;
Seifi, Ali Reza .
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, 2021, 17 (01) :659-666
[2]   Deep Reinforcement Learning A brief survey [J].
Arulkumaran, Kai ;
Deisenroth, Marc Peter ;
Brundage, Miles ;
Bharath, Anil Anthony .
IEEE SIGNAL PROCESSING MAGAZINE, 2017, 34 (06) :26-38
[3]   A Decentralized Electricity Market Scheme Enabling Demand Response Deployment [J].
Bahrami, Shahab ;
Amini, M. Hadi ;
Shafie-khah, Miadreza ;
Catalao, Joao P. S. .
IEEE TRANSACTIONS ON POWER SYSTEMS, 2018, 33 (04) :4218-4227
[4]   Convolutional Neural Networks for Automatic State-Time Feature Extraction in Reinforcement Learning Applied to Residential Load Control [J].
Claessens, Bert J. ;
Vrancx, Peter ;
Ruelens, Frederik .
IEEE TRANSACTIONS ON SMART GRID, 2018, 9 (04) :3259-3269
[5]   Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning [J].
Du, Yan ;
Zandi, Helia ;
Kotevska, Olivera ;
Kurte, Kuldeep ;
Munk, Jeffery ;
Amasyali, Kadir ;
Mckee, Evan ;
Li, Fangxing .
APPLIED ENERGY, 2021, 281
[6]   A Distributed Online Pricing Strategy for Demand Response Programs [J].
Li, Pan ;
Wang, Hao ;
Zhang, Baosen .
IEEE TRANSACTIONS ON SMART GRID, 2019, 10 (01) :350-360
[7]   Real-Time Demand Bidding for Energy Management in Discrete Manufacturing Facilities [J].
Li, Yi-Chang ;
Hong, Seung Ho .
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, 2017, 64 (01) :739-749
[8]   Practical Implementation of an OPC UA TSN Communication Architecture for a Manufacturing System [J].
Li, Yuting ;
Jiang, Junhui ;
Lee, Changdae ;
Hong, Seung Ho .
IEEE ACCESS, 2020, 8 :200100-200111
[9]   Deep Learning Based Densely Connected Network for Load Forecasting [J].
Li, Zhuoling ;
Li, Yuanzheng ;
Liu, Yun ;
Wang, Ping ;
Lu, Renzhi ;
Gooi, Hoay Beng .
IEEE TRANSACTIONS ON POWER SYSTEMS, 2021, 36 (04) :2829-2840
[10]   Data-driven real-time price-based demand response for industrial facilities energy management [J].
Lu, Renzhi ;
Bai, Ruichang ;
Huang, Yuan ;
Li, Yuting ;
Jiang, Junhui ;
Ding, Yuemin .
APPLIED ENERGY, 2021, 283