Approximate dynamic programming approach for process control

被引:46
作者
Lee, Jay H. [1 ]
Wong, Weechin [1 ]
机构
[1] Georgia Inst Technol, Sch Chem & Biomol Engn, Atlanta, GA 30332 USA
关键词
Stochastic process control; Stochastic dynamic programming; Approximate dynamic programming; Dual control; Constrained control; DESIGN; ISSUES;
D O I
10.1016/j.jprocont.2010.06.007
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
We assess the potentials of the approximate dynamic programming (ADP) approach for process control, especially as a method to complement the model predictive control (MPC) approach. In the artificial intelligence (AI) and operations research (OR) research communities, ADP has recently seen significant activities as an effective method for solving Markov decision processes (MDPs), which represent a type of multi-stage decision problems under uncertainty. Process control problems are similar to MDPs with the key difference being the continuous state and action spaces as opposed to discrete ones. In addition, unlike in other popular ADP application areas like robotics or games, in process control applications first and foremost concern should be on the safety and economics of the on-going operation rather than on efficient learning. We explore different options within ADP design, such as the pre-decision state vs. post-decision state value function, parametric vs. nonparametric value function approximator, batch-mode vs. continuous-mode learning, and exploration vs. robustness. We argue that ADP possesses great potentials, especially for obtaining effective control policies for stochastic constrained nonlinear or linear systems and continually improving them towards optimality. (C) 2010 Elsevier Ltd. All rights reserved.
引用
收藏
页码:1038 / 1048
页数:11
相关论文
共 50 条
[41]   Opportunistic Fair Scheduling in Wireless Networks: An Approximate Dynamic Programming Approach [J].
Zhi Zhang ;
Sudhir Moola ;
Edwin K. P. Chong .
Mobile Networks and Applications, 2010, 15 :710-728
[42]   Opportunistic Fair Scheduling in Wireless Networks: An Approximate Dynamic Programming Approach [J].
Zhang, Zhi ;
Moola, Sudhir ;
Chong, Edwin K. P. .
MOBILE NETWORKS & APPLICATIONS, 2010, 15 (05) :710-728
[43]   An approximate dynamic programming approach to project scheduling with uncertain resource availabilities [J].
Xie, Fang ;
Li, Haitao ;
Xu, Zhe .
APPLIED MATHEMATICAL MODELLING, 2021, 97 :226-243
[44]   A kernel-based approximate dynamic programming approach: Theory and application [J].
Forootani, Ali ;
Iervolino, Raffaele ;
Tipaldi, Massimo ;
Baccari, Silvio .
AUTOMATICA, 2024, 162
[45]   Sourcing strategies in supply risk management: An approximate dynamic programming approach [J].
Fang, Jiarui ;
Zhao, Lei ;
Fransoo, Jan C. ;
Van Woensel, Tom .
COMPUTERS & OPERATIONS RESEARCH, 2013, 40 (05) :1371-1382
[46]   Optimal control of heavy haul train based on approximate dynamic programming [J].
Wang, Xi ;
Tang, Tao ;
He, Hui .
ADVANCES IN MECHANICAL ENGINEERING, 2017, 9 (04) :1-15
[47]   Approximate dynamic programming-based control of distributed parameter systems [J].
Joy, Midhun ;
Kaisare, Niket S. .
ASIA-PACIFIC JOURNAL OF CHEMICAL ENGINEERING, 2011, 6 (03) :452-459
[48]   Production control in a complex production system using approximate dynamic programming [J].
Wu, Han ;
Evans, Gerald ;
Bae, Ki-Hwan .
INTERNATIONAL JOURNAL OF PRODUCTION RESEARCH, 2016, 54 (08) :2419-2432
[49]   Insect pest control, approximate dynamic programming, and the management of the evolution of resistance [J].
Hackett, Sean C. ;
Bonsall, Michael B. .
ECOLOGICAL APPLICATIONS, 2019, 29 (02)
[50]   Markov decision process and approximate dynamic programming for a patient assignment scheduling problem [J].
O'Reilly, Malgorzata M. ;
Krasnicki, Sebastian ;
Montgomery, James ;
Heydar, Mojtaba ;
Turner, Richard ;
Van Dam, Pieter ;
Maree, Peter .
ANNALS OF OPERATIONS RESEARCH, 2025, :1493-1531