Continuous-Time Distributed Policy Iteration for Multicontroller Nonlinear Systems

被引：82
作者：
Wei, Qinglai [1 ,2 ,3 ]
Li, Hongyang [1 ,2 ,3 ]
Yang, Xiong [4 ]
He, Haibo [5 ]
机构：
[1] Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
[2] Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China
[3] Qingdao Acad Intelligent Ind, Qingdao 266109, Peoples R China
[4] Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China
[5] Univ Rhode Isl, Dept Elect Comp & Biomed Engn, Kingston, RI 02881 USA
来源：
IEEE TRANSACTIONS ON CYBERNETICS | 2021年 / 51卷 / 05期
基金：
美国国家科学基金会; 中国国家自然科学基金;
关键词：
Optimal control; Nonlinear systems; Decentralized control; Mathematical model; Convergence; Multi-agent systems; Adaptive dynamic programming (ADP); approximate dynamic programming; distributed policy iteration; nonlinear systems; optimal control; ZERO-SUM GAMES; MULTIAGENT SYSTEMS; TRACKING CONTROL; DRIVEN;
D O I：
10.1109/TCYB.2020.2979614
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
In this article, a novel distributed policy iteration algorithm is established for infinite horizon optimal control problems of continuous-time nonlinear systems. In each iteration of the developed distributed policy iteration algorithm, only one controller's control law is updated and the other controllers' control laws remain unchanged. The main contribution of the present algorithm is to improve the iterative control law one by one, instead of updating all the control laws in each iteration of the traditional policy iteration algorithms, which effectively releases the computational burden in each iteration. The properties of distributed policy iteration algorithm for continuous-time nonlinear systems are analyzed. The admissibility of the present methods has also been analyzed. Monotonicity, convergence, and optimality have been discussed, which show that the iterative value function is nonincreasingly convergent to the solution of the Hamilton-Jacobi-Bellman equation. Finally, numerical simulations are conducted to illustrate the effectiveness of the proposed method.
引用
收藏
页码：2372 / 2383
页数：12
相关论文
共 60 条
[1]   Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach [J].
Abu-Khalaf, M ;
Lewis, FL .
AUTOMATICA, 2005, 41 (05) :779-791
[2]  
[Anonymous], 2012, Thesis
[3]   Value and Policy Iterations in Optimal Control and Adaptive Dynamic Programming [J].
Bertsekas, Dimitri P. .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2017, 28 (03) :500-509
[4]   A Connection Between Dynamic Region-Following Formation Control and Distributed Average Tracking [J].
Chen, Fei ;
Ren, Wei .
IEEE TRANSACTIONS ON CYBERNETICS, 2018, 48 (06) :1760-1772
[5]   Minimal-Approximation-Based Distributed Consensus Tracking of a Class of Uncertain Nonlinear Multiagent Systems With Unknown Control Directions [J].
Choi, Yun Ho ;
Yoo, Sung Jin .
IEEE TRANSACTIONS ON CYBERNETICS, 2017, 47 (08) :1994-2007
[6]   Asymptotical Cooperative Tracking Control for Unknown High-Order Multi-Agent Systems via Distributed Adaptive Critic Design [J].
Cui, Lili ;
Li, Ye ;
Yang, Rui ;
Yang, Xin .
IEEE ACCESS, 2018, 6 :24650-24659
[7]   Stability in probability and inverse optimal control of evolution systems driven by Levy processes [J].
Do, Khac Duc .
IEEE-CAA JOURNAL OF AUTOMATICA SINICA, 2020, 7 (02) :405-419
[8]   Distributed Nonlinear Control With Event-Triggered Communication to Achieve Current-Sharing and Voltage Regulation in DC Microgrids [J].
Han, Renke ;
Meng, Lexuan ;
Guerrero, Josep M. ;
Vasquez, Juan C. .
IEEE TRANSACTIONS ON POWER ELECTRONICS, 2018, 33 (07) :6416-6433
[9]   Optimal guaranteed cost control of uncertain non-linear systems using adaptive dynamic programming with concurrent learning [J].
Huang, Yuzhu .
IET CONTROL THEORY AND APPLICATIONS, 2018, 12 (08) :1025-1035
[10]   Neural-Network-Based Robust Control Schemes for Nonlinear Multiplayer Systems With Uncertainties via Adaptive Dynamic Programming [J].
Jiang, He ;
Zhang, Huaguang ;
Luo, Yanhong ;
Han, Ji .
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS, 2019, 49 (03) :579-588
123456