Demand response-based autonomous voltage control for transmission and active distribution networks using modified partially observed Markov decision process model

To fully utilize the voltage regulation capacity and interaction characteristics of the Transmission and Distribution (T&D) system, a novel Modified Partially Observed Markov Decision Process (MPOMDP)-based Reinforcement Learning (RL) scheme for Autonomous Voltage Control is proposed, which incorporates Demand Response (DR) and cooperation with the Transmission Network . The proposed scheme consists of two vital components: an MPOMDP block and a Modified Asynchronous Advantage Actor-Critic-based RL block. The MPOMDP block innovatively exploits the confidence interval of the observed state to make a better perception of the precise system state by introducing two new probability vectors. Then the MPOMDP block is fed into the underlying architecture of the RL block for asynchronously capturing features and optimal decision-making, where the solving framework additionally brings in a public data buffer to realize boundary information sharing. Case studies are conducted on a modified T&D system considering N-1 contingencies, with a training dataset from a district in Suzhou, China. Simulation results demonstrate that the proposed scheme can achieve significant voltage optimization while ensuring fast convergence speed. A novel Modified Partially Observed Markov Decision Process (MPOMDP)-based Reinforcement Learning (RL) scheme for Autonomous Voltage Control is proposed, which incorporates Demand Response and cooperation with the Transmission Network. The proposed scheme consists of two vital components: an MPOMDP block which makes a better perception of the precise system state and a Modified Asynchronous Advantage Actor-Critic-based RL block which assists agents asynchronously to capture features and generate optimal decision-making.image