Robust Optimal Well Control using an Adaptive Multigrid Reinforcement Learning Framework

被引:0
作者
Atish Dixit
Ahmed H. Elsheikh
机构
[1] Heriot-Watt University,
来源
Mathematical Geosciences | 2023年 / 55卷
关键词
Reinforcement learning; Adaptive; Multigrid framework; Transfer learning; Robust optimal control;
D O I
暂无
中图分类号
学科分类号
摘要
Reinforcement learning (RL) is a promising tool for solving robust optimal well control problems where the model parameters are highly uncertain and the system is partially observable in practice. However, the RL of robust control policies often relies on performing a large number of simulations. This could easily become computationally intractable for cases with computationally intensive simulations. To address this bottleneck, an adaptive multigrid RL framework is introduced which is inspired by principles of geometric multigrid methods used in iterative numerical algorithms. RL control policies are initially learned using computationally efficient low-fidelity simulations with coarse grid discretization of the underlying partial differential equations (PDEs). Subsequently, the simulation fidelity is increased in an adaptive manner towards the highest fidelity simulation that corresponds to the finest discretization of the model domain. The proposed framework is demonstrated using a state-of-the-art, model-free policy-based RL algorithm, namely the proximal policy optimization algorithm. Results are shown for two case studies of robust optimal well control problems, which are inspired from SPE-10 model 2 benchmark case studies. Prominent gains in computational efficiency are observed using the proposed framework, saving around 60-70% of the computational cost of its single fine-grid counterpart.
引用
收藏
页码:345 / 375
页数:30
相关论文
共 50 条
[11]   Reinforcement learning-based robust optimal tracking control for disturbed nonlinear systems [J].
Fan, Zhong-Xin ;
Tang, Lintao ;
Li, Shihua ;
Liu, Rongjie .
NEURAL COMPUTING & APPLICATIONS, 2023, 35 (33) :23987-23996
[12]   Training Drift Counteraction Optimal Control Policies Using Reinforcement Learning: An Adaptive Cruise Control Example [J].
Li, Zhaojian ;
Chu, Tianshu ;
Kolmanovsky, Ilya, V ;
Yin, Xiang .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2018, 19 (09) :2903-2912
[13]   Robust Optimal Control of Point-Feet Biped Robots Using a Reinforcement Learning Approach [J].
Hou, Yi-You ;
Lin, Ming-Hung ;
Anjidani, Majid ;
Nik, Hassan Saberi .
IETE JOURNAL OF RESEARCH, 2024, 70 (10) :7831-7846
[14]   Dynamic Event-Triggered Robust Optimal Attitude Control of QUAV Using Reinforcement Learning [J].
Jin, Peng ;
Ma, Qian ;
Xu, Shengyuan .
IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS, 2023, 59 (06) :7798-7807
[15]   Multi-objective optimal control of nonlinear processes using reinforcement learning with adaptive weighting [J].
Wang, Yujia ;
Wang, Zhiyuan ;
Wu, Zhe .
COMPUTERS & CHEMICAL ENGINEERING, 2025, 201
[16]   Optimal control of ship unloaders using reinforcement learning [J].
Scardua, LA ;
Da Cruz, JJ ;
Costa, AHR .
ADVANCED ENGINEERING INFORMATICS, 2002, 16 (03) :217-227
[17]   Robust Optimal Formation Control of Heterogeneous Multi-Agent System via Reinforcement Learning [J].
Lin, Wei ;
Zhao, Wanbing ;
Liu, Hao .
IEEE ACCESS, 2020, 8 (08) :218424-218432
[18]   Multiobjective Reinforcement Learning for Reconfigurable Adaptive Optimal Control of Manufacturing Processes [J].
Dornheim, Johannes ;
Link, Norbert .
2018 13TH INTERNATIONAL SYMPOSIUM ON ELECTRONICS AND TELECOMMUNICATIONS (ISETC), 2018, :97-101
[19]   Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems [J].
Pang, Bo ;
Jiang, Zhong-Ping .
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2023, 68 (04) :2383-2390
[20]   A robust reinforcement learning using the concept of sliding mode control [J].
Obayashi, M. ;
Nakahara, N. ;
Kuremoto, T. ;
Kobayashi, K. .
ARTIFICIAL LIFE AND ROBOTICS, 2009, 13 (02) :526-530