Optimal robust online tracking control for space manipulator in task space using off-policy reinforcement learning

被引:1
作者
Zhuang, Hongji [1 ]
Zhou, Hang [1 ]
Shen, Qiang [1 ]
Wu, Shufan [1 ,2 ]
Razoumny, Vladimir Yu. [2 ]
Razoumny, Yury N. [2 ]
机构
[1] Shanghai Jiao Tong Univ, Shanghai 200240, Peoples R China
[2] Peoples Friendship Univ Russia, RUDN Univ, Moscow 117198, Russia
基金
中国国家自然科学基金;
关键词
Task space; Reinforcement learning; Online tracking control; H-INFINITY CONTROL; ROBOT MANIPULATORS; NONLINEAR-SYSTEMS; TIME-SYSTEMS; APPROXIMATION; STATE;
D O I
10.1016/j.ast.2024.109446
中图分类号
V [航空、航天];
学科分类号
08 ; 0825 ;
摘要
This study addresses the demands for adaptability, uncertainty management, and high performance in the control of space manipulators, and the inadequacies in achieving optimal control and handling external uncertainty in task space in previous research. Based on off-policy reinforcement learning, a model-free and time-efficient method for online robust tracking control in task space is devised. To address the complexity of dynamic equations in task space, a mixed-variable approach is adopted to transform the multivariable coupled time- varying problem into a single-variable problem. Subsequently, the optimal control policy is derived with the disturbance convergence, stability, and optimality of the control method being demonstrated. This marks the first instance of achieving robust optimal tracking control in task space for space manipulators. The efficacy and superiority of the presented algorithm are validated through simulation.
引用
收藏
页数:12
相关论文
共 50 条
[21]   H∞ Optimal Control of Unknown Linear Discrete-time Systems: An Off-policy Reinforcement Learning Approach [J].
Kiumarsi, Bahare ;
Modares, Hamidreza ;
Lewis, Frank L. ;
Jiang, Zhong-Ping .
PROCEEDINGS OF THE 2015 7TH IEEE INTERNATIONAL CONFERENCE ON CYBERNETICS AND INTELLIGENT SYSTEMS (CIS) AND ROBOTICS, AUTOMATION AND MECHATRONICS (RAM), 2015, :41-46
[22]   Safe Off-policy Reinforcement Learning Using Barrier Functions [J].
Marvi, Zahra ;
Kiumarsi, Bahare .
2020 AMERICAN CONTROL CONFERENCE (ACC), 2020, :2176-2181
[23]   Optimal Synchronization Control of Multiagent Systems With Input Saturation via Off-Policy Reinforcement Learning [J].
Qin, Jiahu ;
Li, Man ;
Shi, Yang ;
Ma, Qichao ;
Zheng, Wei Xing .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2019, 30 (01) :85-96
[24]   A Hybrid Online Off-Policy Reinforcement Learning Agent Framework Supported by Transformers [J].
Villarrubia-Martin, Enrique Adrian ;
Rodriguez-Benitez, Luis ;
Jimenez-Linares, Luis ;
Munoz-Valero, David ;
Liu, Jun .
INTERNATIONAL JOURNAL OF NEURAL SYSTEMS, 2023, 33 (12)
[25]   Enhanced Strategies for Off-Policy Reinforcement Learning Algorithms in HVAC Control [J].
Chen, Zhe ;
Jia, Qingshan .
2024 14TH ASIAN CONTROL CONFERENCE, ASCC 2024, 2024, :1691-1696
[26]   Off-Policy Reinforcement Learning: Optimal Operational Control for Two-Time-Scale Industrial Processes [J].
Li, Jinna ;
Kiumarsi, Bahare ;
Chai, Tianyou ;
Lewis, Frank L. ;
Fan, Jialu .
IEEE TRANSACTIONS ON CYBERNETICS, 2017, 47 (12) :4547-4558
[27]   Distributed Optimal Consensus Control for Coupled Linear Systems Based on Off-Policy Integral Reinforcement Learning [J].
Zhao, Wenyan ;
Liu, Zhongchang ;
Yang, Xin .
39TH YOUTH ACADEMIC ANNUAL CONFERENCE OF CHINESE ASSOCIATION OF AUTOMATION, YAC 2024, 2024, :2199-2204
[28]   Motion control of a space manipulator using fuzzy sliding mode control with reinforcement learning [J].
Xie, Zhicheng ;
Sun, Tao ;
Kwan, Trevor ;
Wu, Xiaofeng .
ACTA ASTRONAUTICA, 2020, 176 :156-172
[29]   Off-Policy Meta-Reinforcement Learning With Belief-Based Task Inference [J].
Imagawa, Takahisa ;
Hiraoka, Takuya ;
Tsuruoka, Yoshimasa .
IEEE ACCESS, 2022, 10 :49494-49507
[30]   Model-free H∞ tracking control for de-oiling hydrocyclone systems via off-policy reinforcement learning [J].
Li, Shaobao ;
Durdevic, Petar ;
Yang, Zhenyu .
AUTOMATICA, 2021, 133