MADRL-Based DSO-Customer Coordinated Bi-Level Volt/VAR Optimization Method for Power Distribution Networks

The high penetration of customer-side variable PVs could change power distribution networks (PDNs) operations and lead to frequent nodal voltage violations. The grid-side static var compensators (SVCs), controlled by distribution system operators (DSO), and customer-side PV inverters are usually used as fast devices to mitigate voltage problems caused by PVs. To effectively coordinate these two types of VAR resources with different ownerships, this paper proposes the multi-agent deep reinforcement learning (MADRL) approach, which uses the asymmetric Markov game (ASMG) method to implement a cooperative bi-level Volt/VAR optimization (VVO) framework. In this framework, DSO is the leader, which makes decisions at the upper level to minimize the PDNs energy losses by regulating SVCs, and customers are the followers, which make decisions at the lower level by regulating PV inverters to mitigate nodal voltage deviations. Furthermore, a model-free Bi-level Actor-Critic (Bi-AC) algorithm is proposed to solve the ASMG problem, which defines the agents' decision priorities so that the follower agents can always execute the best response policy. The effectiveness of the proposed Bi-AC method is verified by utilizing improved IEEE 33-bus and IEEE 118-bus case with practical grid operation data.