Real-time energy flexibility optimization of grid-connected smart building communities with deep reinforcement learning

Nowadays, the presence of electric vehicles (EVs) in power distribution networks (DNs) is increasing significantly, where these facilities' smart charging and discharging are mandatory. In response to this challenge, strategic control of EV charging/discharging power can improve power system flexibility and reduce the underlying operation costs. Real-time charging and discharging of EVs is a time-consuming and complicated problem that might suffer from uncertainties in the behavior of EV owners. Respecting the potential to provide fast responses in complex environments, state-of-the-art deep reinforcement learning (DRL) methods can be an appropriate solution for EVs' real-time charging and discharging. This paper studies the application of DRL to real-time energy scheduling of autonomous smart building communities (SBCs) integrated with EV parking lots (EVPLs). A model-free DRL approach based on a twin delayed deep deterministic policy gradient (TD3) algorithm is adopted to reach a near-optimal solution for autonomous SBCs' real-time energy scheduling problem. In addition, a convex mathematical optimal power flow (OPF) is developed to guarantee DN's reliable operation. The findings reflect that real-time strategic charging and discharging of EVs can enhance the flexibility of DN in order to provide energy flexibility in the real-time electricity market.