Reinforcement learning based iterative learning control for nonlinear batch process with non-repetitive uncertainty via Koopman operator

被引:0
作者
Tao, Hongfeng [1 ]
Huang, Yuan [1 ]
Liu, Tao [2 ]
Paszke, Wojciech [3 ]
机构
[1] Jiangnan Univ, Key Lab Adv Proc Control Light Ind, Minist Educ, Wuxi 214000, Peoples R China
[2] Dalian Univ Technol, Inst Adv Control Technol, Dalian 116024, Peoples R China
[3] Univ Zielona Gora, Inst Automat Elect & Elect Engn, Ul Szafrana 2, PL-65246 Zielona Gora, Poland
基金
中国国家自然科学基金;
关键词
Iterative learning control; Nonlinear batch process; Koopman operator; Deep reinforcement learning; Non-repetitive uncertainty; SYSTEMS;
D O I
10.1016/j.jprocont.2025.103402
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
To tackle the time and batchwise uncertainty often involved in nonlinear batch process, this paper proposes a deep reinforcement learning (DRL) based ILC scheme via Koopman operator. Using the Koopman operator, the original nonlinear system is reformulated into a high-dimensional linear space form. Then, a DRL agent with neural network is introduced into the 2D ILC framework to compensate for non-repetitive uncertainty. Correspondingly, a synthetic 2D ILC-DRL scheme is designed to improve the system tracking performance against time and batchwise uncertainty. Meanwhile, the convergence conditions of the proposed ILC scheme are analyzed with a proof through the linear matrix inequality. An illustrative example of continuous stirring tank reactor (CSTR) demonstrates that the established high-dimensional linear model can ensure good accuracy compared to the original nonlinear process model, with an output error of smaller than 5%. Moreover, the tracking error is significantly reduced over 90% by the reinforcement learning based ILC in comparison with the recently developed dynamic iterative linearization and PD-type ILC methods.
引用
收藏
页数:11
相关论文
共 43 条
[1]  
Bao Jiao, 2016, Adv. Model. Anal. B, V59, P113
[2]  
Brunton S. L., 2021, arXiv
[3]   Machine learning based iterative learning control for non-repetitive time-varying systems [J].
Chen, Yiyang ;
Jiang, Wei ;
Charalambous, Themistoklis .
INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL, 2023, 33 (07) :4098-4116
[4]   Neural network compensator-based robust iterative learning control scheme for mobile robots nonlinear systems with disturbances and uncertain parameters [J].
Chen, Zhengquan ;
Hou, Yandong ;
Huang, Ruirui ;
Cheng, Qianshuai .
APPLIED MATHEMATICS AND COMPUTATION, 2024, 469
[5]   Convergence Analysis of Sampled-Data ILC for Locally Lipschitz Continuous Nonlinear Nonaffine Systems With Nonrepetitive Uncertainties [J].
Chi, Ronghu ;
Hui, Yu ;
Chien, Chiang-Ju ;
Huang, Biao ;
Hou, Zhongsheng .
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2021, 66 (07) :3347-3354
[6]   Computationally Efficient Data-Driven Higher Order Optimal Iterative Learning Control [J].
Chi, Ronghu ;
Hou, Zhongsheng ;
Jin, Shangtai ;
Huang, Biao .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2018, 29 (12) :5971-5980
[7]   Data-driven high-order terminal iterative learning control with a faster convergence speed [J].
Chi, Ronghu ;
Huang, Biao ;
Hou, Zhongsheng ;
Jin, Shangtai .
INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL, 2018, 28 (01) :103-119
[8]  
Chou PW, 2017, PR MACH LEARN RES, V70
[9]  
Dargazany A, 2021, Arxiv, DOI arXiv:2105.13806
[10]   Averaged Soft Actor-Critic for Deep Reinforcement Learning [J].
Ding, Feng ;
Ma, Guanfeng ;
Chen, Zhikui ;
Gao, Jing ;
Li, Peng .
COMPLEXITY, 2021, 2021