Neural network-based event-triggered integral reinforcement learning for constrained H1 tracking control with experience replay

被引:14
作者
Xue, Shan [1 ,2 ]
Luo, Biao [3 ]
Liu, Derong [4 ]
Gao, Ying [1 ]
机构
[1] South China Univ Technol, Sch Comp Sci & Engn, Guangzhou 510006, Peoples R China
[2] Peng Cheng Lab, Shenzhen 518000, Peoples R China
[3] Cent South Univ, Sch Automat, Changsha 410083, Peoples R China
[4] Univ Illinois, Dept Elect & Comp Engn, Chicago, IL 60607 USA
关键词
Adaptive dynamic programming; Neural networks; Integral reinforcement learning; H 1 tracking control; Event -triggered mechanism; UNCERTAIN NONLINEAR-SYSTEMS; FIXED-TIME CONSENSUS; POLICY ITERATION; FEEDBACK-CONTROL; ALGORITHM; DESIGN;
D O I
10.1016/j.neucom.2022.09.119
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Since input constraints and external disturbances are unavoidable in tracking control problems, how to obtain a controller in this case to save communication and data resources at the same time is very chal-lenging. Aiming at these challenges, this paper develops a novel neural network (NN)-based event -triggered integral reinforcement learning (IRL) algorithm for constrained H1 tracking control problems. First, the constrained H1 tracking control problem is transformed into a regulation problem. Second, an event-triggered optimal controller is designed to reduce network transmission burden and improve resource utilization, where a novel threshold is proposed and its non-negativity can be guaranteed. Third, for implementation purpose, a novel NN-based event-triggered IRL algorithm is developed. In order to improve data utilization, the experience replay technique with an easy-to-verify condition is employed in the learning process. Theoretical analysis proves that the tracking error and weight estima-tion error are uniformly ultimately bounded. Finally, simulation verification shows the effectiveness of the present method. (c) 2022 Elsevier B.V. All rights reserved.
引用
收藏
页码:25 / 35
页数:11
相关论文
共 62 条
[1]  
Abu-Khalaf M., 2006, NONLINEAR H2 H1 CONS
[2]   Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof [J].
Al-Tamimi, Asma ;
Lewis, Frank L. ;
Abu-Khalaf, Murad .
IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS, 2008, 38 (04) :943-949
[3]   Event-Triggered Multigradient Recursive Reinforcement Learning Tracking Control for Multiagent Systems [J].
Bai, Weiwei ;
Li, Tieshan ;
Long, Yue ;
Chen, C. L. Philip .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2023, 34 (01) :366-379
[4]   Concurrent Learning for Convergence in Adaptive Control without Persistency of Excitation [J].
Chowdhary, Girish ;
Johnson, Eric .
49TH IEEE CONFERENCE ON DECISION AND CONTROL (CDC), 2010, :3674-3679
[5]   Event-Triggered Adaptive Dynamic Programming for Continuous-Time Systems With Control Constraints [J].
Dong, Lu ;
Zhong, Xiangnan ;
Sun, Changyin ;
He, Haibo .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2017, 28 (08) :1941-1952
[6]   Reinforcement learning in continuous time and space [J].
Doya, K .
NEURAL COMPUTATION, 2000, 12 (01) :219-245
[7]   Online policy iterative-based H∞ optimization algorithm for a class of nonlinear systems [J].
He, Shuping ;
Fang, Haiyang ;
Zhang, Maoguang ;
Liu, Fei ;
Luan, Xiaoli ;
Ding, Zhengdao .
INFORMATION SCIENCES, 2019, 495 :1-13
[8]   Adaptive Optimal Control for a Class of Nonlinear Systems: The Online Policy Iteration Approach [J].
He, Shuping ;
Fang, Haiyang ;
Zhang, Maoguang ;
Liu, Fei ;
Ding, Zhengtao .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2020, 31 (02) :549-558
[9]  
Khalil H. K., 1996, Nonlinear systems
[10]   Reinforcement Learning and Feedback Control USING NATURAL DECISION METHODS TO DESIGN OPTIMAL ADAPTIVE CONTROLLERS [J].
Lewis, Frank L. ;
Vrabie, Draguna ;
Vamvoudakis, Kyriakos G. .
IEEE CONTROL SYSTEMS MAGAZINE, 2012, 32 (06) :76-105