A novel optimization framework for natural gas transportation pipeline networks based on deep reinforcement learning

Natural gas is an emerging and reliable energy source in transition to a low-carbon economy. The natural gas transportation pipeline network systems are crucial when transporting natural gas from the production endpoints to processing or consuming endpoints. Optimizing the operational efficiency of compressor stations within pipeline networks is an effective way to reduce energy consumption and carbon emissions during transportation. This paper proposes an optimization framework for natural gas transportation pipeline networks based on deep reinforcement learning (DRL). The mathematical simulation model is derived from mass balance, hydrodynamics principles of gas flow, and compressor characteristics. The optimization control problem in steady state is formulated into a one-step Markov decision process (MDP) and solved by DRL. The decision variables are selected as the discharge ratio of each compressor. By the comprehensive comparison with dynamic programming (DP) and genetic algorithm (GA) in three typical element topologies (a linear topology with gun-barrel structure, a linear topology with branch structure, and a tree topology), the proposed method can obtain 4.60% lower power consumption than GA, and the time consumption is reduced by 97.5% compared with DP. The proposed framework could be further utilized for future large-scale network optimization practices.