Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints

被引:169
作者
Yang, Xiong [1 ]
Liu, Derong [1 ]
Wang, Ding [1 ]
机构
[1] Chinese Acad Sci, State Key Lab Management & Control Complex Syst, Inst Automat, Beijing 100190, Peoples R China
基金
中国国家自然科学基金;
关键词
adaptive control; input constraints; neural networks; optimal control; reinforcement learning; ARCHITECTURE; ALGORITHM; DESIGN;
D O I
10.1080/00207179.2013.848292
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In this paper, an adaptive reinforcement learning-based solution is developed for the infinite-horizon optimal control problem of constrained-input continuous-time nonlinear systems in the presence of nonlinearities with unknown structures. Two different types of neural networks (NNs) are employed to approximate the Hamilton-Jacobi-Bellman equation. That is, an recurrent NN is constructed to identify the unknown dynamical system, and two feedforward NNs are used as the actor and the critic to approximate the optimal control and the optimal cost, respectively. Based on this framework, the action NN and the critic NN are tuned simultaneously, without the requirement for the knowledge of system drift dynamics. Moreover, by using Lyapunov's direct method, the weights of the action NN and the critic NN are guaranteed to be uniformly ultimately bounded, while keeping the closed-loop system stable. To demonstrate the effectiveness of the present approach, simulation results are illustrated.
引用
收藏
页码:553 / 566
页数:14
相关论文
共 50 条
[41]   Reinforcement learning for robust adaptive control of partially unknown nonlinear systems subject to unmatched uncertainties [J].
Yang, Xiong ;
He, Haibo ;
Wei, Qinglai ;
Luo, Biao .
INFORMATION SCIENCES, 2018, 463 :307-322
[42]   Adaptive Event-Triggered Near-Optimal Tracking Control for Unknown Continuous-Time Nonlinear Systems [J].
Wang, Kunfu ;
Gu, Qijia ;
Huang, Baiqiao ;
Wei, Qinglai ;
Zhou, Tianmin .
IEEE ACCESS, 2022, 10 :9506-9518
[43]   Reinforcement learning-based optimal control of unknown constrained-input nonlinear systems using simulated experience [J].
Asl, Hamed Jabbari ;
Uchibe, Eiji .
NONLINEAR DYNAMICS, 2023, 111 (17) :16093-16110
[44]   Reinforcement learning-based optimal control of unknown constrained-input nonlinear systems using simulated experience [J].
Hamed Jabbari Asl ;
Eiji Uchibe .
Nonlinear Dynamics, 2023, 111 :16093-16110
[45]   Optimal Control of Nonlinear Continuous-Time Systems in Strict-Feedback Form [J].
Zargarzadeh, Hassan ;
Dierks, Travis ;
Jagannathan, Sarangapani .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2015, 26 (10) :2535-2549
[46]   Data-Based Self-Learning Optimal Control for Continuous-Time Unknown Nonlinear Systems With Disturbance [J].
Wei, Qinglai ;
Liu, Derong ;
Song, Ruizhuo ;
Yan, Pengfei .
PROCEEDINGS OF THE 28TH CHINESE CONTROL AND DECISION CONFERENCE (2016 CCDC), 2016, :6633-6638
[47]   Online approximate solution of HJI equation for unknown constrained-input nonlinear continuous-time systems [J].
Yang, Xiong ;
Liu, Derong ;
Ma, Hongwen ;
Xu, Yancai .
INFORMATION SCIENCES, 2016, 328 :435-454
[48]   Constrained Reinforcement Learning-Based Closed-Loop Reference Model for Optimal Tracking Control of Unknown Continuous-Time Systems [J].
Zhang, Haoran ;
Zhao, Chunhui ;
Ding, Jinliang .
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, 2024, 21 (04) :7312-7324
[49]   Reinforcement learning-based event-triggered optimal control for unknown nonlinear systems with input delay [J].
Chen, Xiangyu ;
Sun, Weiwei ;
Gao, Xinci ;
Li, Yongshu .
INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL, 2024, 34 (07) :4844-4863
[50]   Inverse Reinforcement Learning of Nonlinear Systems with Input Constraints [J].
You, Weijie ;
Zhang, Huaipin ;
Zhao, Wei .
PROCEEDINGS OF THE 36TH CHINESE CONTROL AND DECISION CONFERENCE, CCDC 2024, 2024, :4084-4088