Transfer learning for occupancy-based HVAC control: A data-driven approach using unsupervised learning of occupancy profiles and deep reinforcement learning

被引:12
作者
Esrafilian-Najafabadi, Mohammad [1 ]
Haghighat, Fariborz [1 ]
机构
[1] Concordia Univ, Dept Bldg Civil & Environm Engn, Energy & Environm Grp, Montreal, PQ, Canada
关键词
Deep reinforcement learning; HVAC control; Transfer learning; Occupancy patterns; Energy efficiency; ENERGY; PREDICTION; THERMOSTATS;
D O I
10.1016/j.enbuild.2023.113637
中图分类号
TU [建筑科学];
学科分类号
0813 ;
摘要
Model-free heating, ventilation, and air conditioning (HVAC) control systems have demonstrated promising potential for adjusting indoor setpoint temperature based on dynamic occupancy patterns in smart buildings. Although these control systems offer the advantage of not needing building or occupancy models, the involved trial-and-error learning process can cause considerable thermal discomfort for occupants, particularly during the initial learning period. Given the critical importance of thermal comfort, this limitation is a major barrier to the practical implementation of such systems. To address this challenge, the present study proposes a framework to enhance the learning process of the model-free HVAC controllers. Specifically, a transfer learning (TL) technique is adopted based on a similarity analysis of occupancy patterns using an unsupervised learning of occupancy profiles. This control framework leverages a k-means clustering algorithm with dynamic time warping to match the most similar households in terms of occupancy patterns within 26 residential units. The results demonstrate that the proposed method significantly improves the performance of the HVAC control system. It enhances the jumpstart performance and total rewards by nearly 25% and 5%, respectively, compared to a conventional model-free controller. Furthermore, it reduces the deviation period and mean temperature deviation by approximately 4% and 68%, respectively. Overall, this framework presents a promising approach to enhancing the performance and practicality of model-free HVAC control systems by reducing the thermal discomfort during the learning process.
引用
收藏
页数:16
相关论文
共 50 条
[41]   Data-Driven Super-Parameterization Using Deep Learning: Experimentation With Multiscale Lorenz 96 Systems and Transfer Learning [J].
Chattopadhyay, Ashesh ;
Subel, Adam ;
Hassanzadeh, Pedram .
JOURNAL OF ADVANCES IN MODELING EARTH SYSTEMS, 2020, 12 (11)
[42]   Reinforcement Learning Based Monitoring and Control of Indoor Carbon Dioxide Concentration Integrating Occupancy Presence [J].
Xie, Xiang ;
Lu, Qiuchen ;
Parlikad, Ajith Kumar ;
Puri, Ramprakash Srinivasan .
CONSTRUCTION RESEARCH CONGRESS 2020: COMPUTER APPLICATIONS, 2020, :258-267
[43]   Data-driven virtual sensor for powertrains based on transfer learning [J].
Karhinen, Aku ;
Hamalainen, Aleksanteri ;
Manngard, Mikael ;
Miettinen, Jesse ;
Viitala, Raine .
BULLETIN OF THE POLISH ACADEMY OF SCIENCES-TECHNICAL SCIENCES, 2023, 71 (06)
[44]   RLclean: An unsupervised integrated data cleaning framework based on deep reinforcement learning [J].
Peng, Jinfeng ;
Shen, Derong ;
Nie, Tiezheng ;
Kou, Yue .
INFORMATION SCIENCES, 2024, 682
[45]   Data analytics for occupancy pattern learning to reduce the energy consumption of HVAC systems in office buildings [J].
Capozzoli, Alfonso ;
Piscitelli, Marco Savino ;
Gorrino, Alice ;
Ballarini, Ilaria ;
Corrado, Vincenzo .
SUSTAINABLE CITIES AND SOCIETY, 2017, 35 :191-208
[46]   Research and Application of Predictive Control Method Based on Deep Reinforcement Learning for HVAC Systems [J].
Fu, Chenhui ;
Zhang, Yunhua .
IEEE ACCESS, 2021, 9 (130845-130852) :130845-130852
[47]   Optimal control method of HVAC based on multi-agent deep reinforcement learning [J].
Fu, Qiming ;
Chen, Xiyao ;
Ma, Shuai ;
Fang, Nengwei ;
Xing, Bin ;
Chen, Jianping .
ENERGY AND BUILDINGS, 2022, 270
[48]   Transfer Learning-Based Deep Reinforcement Learning Approach for Robust Route Guidance in Mixed Traffic Environment [J].
Lee, Donghoun .
IEEE ACCESS, 2024, 12 :61667-61680
[49]   Experience-driven Networking: A Deep Reinforcement Learning based Approach [J].
Xu, Zhiyuan ;
Tang, Jian ;
Meng, Jingsong ;
Zhang, Weiyi ;
Wang, Yanzhi ;
Liu, Chi Harold ;
Yang, Dejun .
IEEE CONFERENCE ON COMPUTER COMMUNICATIONS (IEEE INFOCOM 2018), 2018, :1880-1888
[50]   Whole building energy model for HVAC optimal control: A practical framework based on deep reinforcement learning [J].
Zhang, Zhiang ;
Chong, Adrian ;
Pan, Yuqi ;
Zhang, Chenlu ;
Lam, Khee Poh .
ENERGY AND BUILDINGS, 2019, 199 :472-490