Operational optimization for off-grid renewable building energy system using deep reinforcement learning

With the application of renewable energy in single office buildings, an increasing number of power grids require building systems coupled with renewable energy to realize off-grid operation. However, the uncertainty of renewable energy sources and the safety of the corresponding energy storage equipment have become major challenges for these systems. Reinforcement learning has made considerable progress in the field of building control as an advanced control algorithm; however, research on its application to the off-grid operation of renewable energy systems, particularly the specific reward function design is limited. Therefore, this study considered the off-grid operation of a renewable building energy system and the safety (prevention of deterioration) of its battery as optimization goals. This study is based on a real building energy system, through the operation control of generators, solar photovoltaics, and batteries to achieve optimization purposes. Aiming at these two optimization goals, this study introduces a detailed reward function design and complete reinforcement learning workflow through a Gaussian distribution. Two deep reinforcement learning (DRL) algorithms were trained and verified by offline reinforcement learning based on the measured data-sets of actual existing buildings in Japan. The results show that the proposed reinforcement learning design can better achieve the two optimization goals of off-grid operation and battery safety under ordinary and extreme conditions. On off-grid operation tasks, the best DRL algorithm can achieve a mean hourly grid power purchase error of less than 2 kWh for the entire optimization window. Furthermore, the proposed deep reinforcement learning algorithms can simultaneously maintain a maximum average 7.72 h of battery unsafe state over the entire 168-hour optimization window.