Human-Machine Collaborative Reinforcement Learning for Power Line Flow Regulation

The complexity and uncertainty in power systems leads to a great challenge for controlling the power grid using traditional manual adjustment methods. Reinforcement learning is a promising data-driven paradigm to address control issues in power grids. This article presents a novel human-machine collaborative (HMC) framework for line flow control. We formulate the collaboration between humans and machines as an extended Markov decision process (MDP) and introduce a human-machine collaborative reinforcement learning (HMC-RL) approach, which comprises a routing module, a machine dispatching module and an HMC dispatching module. The routing module determines whether the power system should be operated by the machine or through human-machine collaboration. The machine dispatching module predicts a machine dispatching action for regulating line flow, while the HMC module predicts an HMC dispatching action with human assistance. Experimental results conducted on the IEEE 39-bus and IEEE 118-bus systems demonstrate that our HMC-RL approach can significantly improve the performance of regulation compared to the machine dispatching policy. Specifically, HMC-RL achieves a 40.03% performance improvement on the IEEE 118-bus system, with 25.8% of human participation.