Explicit Reinforcement Learning Safety Layer for Computationally Efficient Inverter-Based Voltage Regulation

To mitigate fast voltage fluctuations caused by high penetration of renewable energy, efficient control, and coordination methods to utilize the reactive power support of inverters are required. Capturing the nonlinear power flow dynamics while enforcing the feasibility of safety constraints, reinforcement learning (RL) with a safety layer shows a promising future in safety-critical voltage regulation tasks. This paper proposes an explicit Deep RL safety layer to achieve computationally efficient voltage regulation of distribution grids with guaranteed hard constraints of voltage security. To achieve this, we first construct the explicit form of the safety layer via an offline search based on multiparametric programming. Then, instead of doing an exhaustive search with exponential complexity, we propose a sample-based approach to identify active constraint sets relevant to safe operations, which makes the offline construction tractable even for large-scale systems. An end-to-end trainable and computationally efficient safe reinforcement learning approach for voltage regulation is proposed based on the explicit safety layer. The performance and computational efficiency of the proposed method are verified by the case study.