Reinforcement Learning Formation Tracking of Networked Autonomous Surface Vehicles With Bounded Inputs via Cloud-Supported Communication

被引:4
作者
Ding, Teng-Fei [1 ]
Ge, Ming-Feng [1 ]
Liu, Zhi-Wei [2 ]
Wang, Leimin [3 ]
Liu, Jie [4 ]
机构
[1] China Univ Geosci, Sch Mech Engn & Elect Informat, Wuhan 430074, Peoples R China
[2] Huazhong Univ Sci & Technol, Sch Artificial Intelligence & Automat, Wuhan 430074, Peoples R China
[3] China Univ Geosci, Sch Automat, Wuhan 430074, Peoples R China
[4] Huazhong Univ Sci & Technol, Sch Civil & Hydraul Engn, Wuhan 430074, Peoples R China
来源
IEEE TRANSACTIONS ON INTELLIGENT VEHICLES | 2024年 / 9卷 / 01期
基金
中国国家自然科学基金;
关键词
Actuators; Vehicle dynamics; Reinforcement learning; Costs; Monitoring; Target tracking; Stability criteria; Networked autonomous surface vehicles (NASVs); reinforcement learning; formation tracking; bounded inputs; cloud-supported communication; CONTAINMENT CONTROL; PERFORMANCE;
D O I
10.1109/TIV.2023.3323767
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
This article investigates formation tracking (FT) problem of the networked autonomous surface vehicles (NASVs) with bounded inputs. In order to achieve distributed control, a prescribed-time observer is employed to reshape the leader's states for the follower ASVs, which can only receive the message from the neighbor ASVs. For reducing communication costs and the negative effect of bounded inputs and the unknown uncertainties, a hierarchical reinforcement learning control (HRLC) algorithm based on the cloud-supported communication is proposed, where the cloud-supported estimator is constructed such that the estimated states approach the leader's states with the less communication costs. The local reinforcement learning controller is designed according to the actor-critic strategy such that the actual states converge to the estimated states with the given formation offset. With the help of Lyapunov stability and Hurwitz stability theory, some sufficient conditions of the close-loop system have be obtained. Finally, simulation examples have be proposed to validate the theoretical analysis.
引用
收藏
页码:469 / 480
页数:12
相关论文
共 42 条
[1]   Cloud-Supported Formation Control of Second-Order Multiagent Systems [J].
Adaldo, Antonio ;
Liuzza, Davide ;
Dimarogonas, Dimos V. ;
Johansson, Karl H. .
IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, 2018, 5 (04) :1563-1574
[2]  
Bing Sun, 2012, 2012 UKACC International Conference on Control (CONTROL), P644, DOI 10.1109/CONTROL.2012.6334705
[3]   Consensus of Multiagent Systems Via Asynchronous Cloud Communication [J].
Bowman, Sean L. ;
Nowzari, Cameron ;
Pappas, George J. .
IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, 2020, 7 (02) :627-637
[4]   Reinforcement Learning-Based Fixed-Time Trajectory Tracking Control for Uncertain Robotic Manipulators With Input Saturation [J].
Cao, Shengjie ;
Sun, Liang ;
Jiang, Jingjing ;
Zuo, Zongyu .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2023, 34 (08) :4584-4595
[5]   Distributed Containment Control for Multiple Autonomous Vehicles With Double-Integrator Dynamics: Algorithms and Experiments [J].
Cao, Yongcan ;
Stuart, Daniel ;
Ren, Wei ;
Meng, Ziyang .
IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, 2011, 19 (04) :929-938
[6]  
Chen HY, 2023, IEEE T IND INFORM, V19, P10034, DOI [10.1109/TII.2022.3232768, 10.1109/TAFFC.2023.3265653]
[7]   Probabilistic Event-Triggered Policy for Extended Dissipative Finite-Time Control of MJS']JSs under Cyber-Attacks and Actuator Failures [J].
Chen, Haiyang ;
Zong, Guangdeng ;
Gao, Fangzheng ;
Shi, Yang .
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2023, 68 (12) :7803-7810
[8]   Cooperative Learning-Based Formation Control of Autonomous Marine Surface Vessels With Prescribed Performance [J].
Dai, Shi-Lu ;
He, Shude ;
Ma, Yufei ;
Yuan, Chengzhi .
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS, 2022, 52 (04) :2565-2577
[9]   Fast Fixed-Time Output Multi-Formation Tracking of Networked Autonomous Surface Vehicles: A Mathematical Induction Method [J].
Ding, Teng-Fei ;
Xu, Kun-Ting ;
Ge, Ming-Feng ;
Park, Ju H. ;
Liang, Chang-Duo .
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2023, 72 (05) :5769-5781
[10]   Prescribed-time formation tracking of second-order multi-agent networks with directed graphs [J].
Ding, Teng-Fei ;
Ge, Ming-Feng ;
Xiong, Caihua ;
Liu, Zhi-Wei ;
Ling, Guang .
AUTOMATICA, 2023, 152