Using Q-Learning for OLTC Voltage Regulation in PV-Rich Distribution Networks

Many electrical distribution networks around the world are already experiencing the massive integration of distributed energy resources, in particular, residential solar photovoltaic (PV) systems. As networks become more observable and controllable, coordinated and active management of all available assets will be required to prevent high PV penetrations from affecting network integrity. This paper explores the benefits and implementation challenges of using Q-learning, a Reinforcement Learning technique, to actively control the on-load tap changer (OLTC) at primary substations and, thus, mitigate voltage rise issues in PV-rich distribution networks. The training process occurs in both offline and online modes. Furthermore, some adjustments are proposed to improve scalability. As a case study, a real Brazilian three-phase MV/LV distribution network with 5,000+ customers is used considering a high PV penetration. Controlling only the OLTC, results demonstrate the importance of adequate reward functions (targeted at reducing voltage problems) and size of the training dataset.