Graph-based Multi-Agent Safe Reinforcement Learning for Voltage Control in Distribution Grids

The growing penetration of distributed energy resources (DERs) has posed a significant challenge in voltage control within modern power distribution networks. Recent research exhibits that advanced Volt-VAR control (VVC) techniques based on deep reinforcement learning (DRL) can effectively stabilize distribution grid voltages by regulating the reactive power output of DERs, employing corrective and preventive measures. However, existing DRL approaches don't consider the network's topology, and therefore, they lack sufficient learning efficiencies. This paper proposes a graph convolution network (GCN) assisted multi-agent deep reinforcement learning (MADRL) based VVC for achieving the optimal control policy for geographically spread inverters and regulating the network nodal voltages using the spatial relationship among system states. It offers a multi-agent soft actor-critic (MASAC) algorithm that involves graph layers in local actor networks that take optimal control actions with the unpredictable generation from the DERs as well as with the topological variations of the distribution networks. Also, it incorporates an exponential barrier function as the reward function, which ensures the safe operation of the DRL agents. The effectiveness of the proposed method is validated on a modified IEEE 34 bus system with various load and PV profiles. It is then compared with a fully connected neural network (FCNN)-based MADRL-based method and a base case scenario with no control actions taken by the agents. The results demonstrate that the proposed GCN-based MADRL method outperforms other methods for regulating the voltages of the network.