Reinforcement learning for natural gas pipeline pressure control

In this research, an off-policy Q-learning algorithm is designed to solve the pressure tracking problem in gas pipelines by adjusting the rotating speed using only the observed data along the system trajectories. How to determine the most appropriate rotational speed by off-policy Q-learning and enforce the pipeline pressure to follow a desired value is very challenging due to nonlinear and unknown dynamics of gas pipeline pressure systems, as well as the requirement of variation of rotational speed. To this end, an optimal control problem of gas pipeline pressure tracking control is first formulated, then an off-policy Q-function based on iterative Bellman equation which is followed by an off-policy Q-learning algorithm used for searching the optimal rotational speed based on reinforcement learning technique and dynamic programming theory, such that the pressure can follow the desired value successfully. The simulation results are shown to validate the effectiveness of the proposed strategy.