Reinforcement learning and adaptive optimization of a class of Markov jump systems with completely unknown dynamic information

被引:55
|
作者
He, Shuping [1 ,2 ]
Zhang, Maoguang [1 ]
Fang, Haiyang [1 ]
Liu, Fei [3 ]
Luan, Xiaoli [3 ]
Ding, Zhengtao [4 ]
机构
[1] Anhui Univ, Sch Elect Engn & Automat, Hefei 230601, Peoples R China
[2] Anhui Univ, Inst Phys Sci & Informat Technol, Hefei 230601, Peoples R China
[3] Jiangnan Univ, Inst Automat, Key Lab Adv Proc Control Light Ind, Minist Educ, Wuxi 214122, Jiangsu, Peoples R China
[4] Univ Manchester, Sch Elect & Elect Engn, Manchester M13 9PL, Lancs, England
基金
中国国家自然科学基金;
关键词
Markov jump linear systems (MJLSs); Adaptive optimal control; Online; Reinforcement learning (RL); Coupled algebraic Riccati equations (AREs); DISCRETE-TIME-SYSTEMS; SLIDING MODE CONTROL; DESIGN; ALGORITHM;
D O I
10.1007/s00521-019-04180-2
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper, an online adaptive optimal control problem of a class of continuous-time Markov jump linear systems (MJLSs) is investigated by using a parallel reinforcement learning (RL) algorithm with completely unknown dynamics. Before collecting and learning the subsystems information of states and inputs, the exploration noise is firstly added to describe the actual control input. Then, a novel parallel RL algorithm is used to parallelly compute the correspondingNcoupled algebraic Riccati equations by online learning. By this algorithm, we will not need to know the dynamic information of the MJLSs. The convergence of the proposed algorithm is also proved. Finally, the effectiveness and applicability of this novel algorithm is illustrated by two simulation examples.
引用
收藏
页码:14311 / 14320
页数:10
相关论文
共 50 条
  • [1] Reinforcement learning and adaptive optimization of a class of Markov jump systems with completely unknown dynamic information
    Shuping He
    Maoguang Zhang
    Haiyang Fang
    Fei Liu
    Xiaoli Luan
    Zhengtao Ding
    Neural Computing and Applications, 2020, 32 : 14311 - 14320
  • [2] Reinforcement learning-based optimal control for Markov jump systems with completely unknown dynamics
    Shi, Xiongtao
    Li, Yanjie
    Du, Chenglong
    Chen, Chaoyang
    Zong, Guangdeng
    Gui, Weihua
    AUTOMATICA, 2025, 171
  • [3] Fuzzy-Based Adaptive Optimization of Unknown Discrete-Time Nonlinear Markov Jump Systems With Off-Policy Reinforcement Learning
    Fang, Haiyang
    Tu, Yidong
    Wang, Hai
    He, Shuping
    Liu, Fei
    Ding, Zhengtao
    Cheng, Shing Shin
    IEEE TRANSACTIONS ON FUZZY SYSTEMS, 2022, 30 (12) : 5276 - 5290
  • [4] Adaptive optimization algorithm for nonlinear Markov jump systems with partial unknown dynamics
    Fang, Haiyang
    Zhu, Guozheng
    Stojanovic, Vladimir
    Nie, Rong
    He, Shuping
    Luan, Xiaoli
    Liu, Fei
    INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL, 2021, 31 (06) : 2126 - 2140
  • [5] Optimal tracking control for completely unknown nonlinear discrete-time Markov jump systems using data-based reinforcement learning method
    Jiang, He
    Zhang, Huaguang
    Luo, Yanhong
    Wang, Junyi
    NEUROCOMPUTING, 2016, 194 : 176 - 182
  • [6] Reinforcement learning-based composite suboptimal control for Markov jump singularly perturbed systems with unknown dynamics
    Li, Wenqian
    Jia, Guolong
    Wang, Yun
    Su, Lei
    Shen, Hao
    MATHEMATICAL METHODS IN THE APPLIED SCIENCES, 2024, 47 (14) : 11551 - 11564
  • [7] Reinforcement learning-based linear quadratic tracking control for partially unknown Markov jump singular interconnected systems
    Jia, Guolong
    Yang, Qing
    Liu, Jinxu
    Shen, Hao
    APPLIED MATHEMATICS AND COMPUTATION, 2025, 491
  • [8] Tracking control optimization scheme for a class of partially unknown fuzzy systems by using integral reinforcement learning architecture
    Zhang, Kun
    Zhang, Huaguang
    Mu, Yunfei
    Sun, Shaoxin
    APPLIED MATHEMATICS AND COMPUTATION, 2019, 359 : 344 - 356
  • [9] Reinforcement Learning-Based Near Optimization for Continuous-Time Markov Jump Singularly Perturbed Systems
    Wang, Jing
    Peng, Chuanjun
    Park, Ju H.
    Shen, Hao
    Shi, Kaibo
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS, 2023, 70 (06) : 2026 - 2030
  • [10] Robust Fault Detection of Nonlinear Singular Markov Jump Systems with Partially Unknown Information
    Shi, Jiangbin
    Yin, Yanyan
    Liu, Yanqing
    Liu, Fei
    PROCEEDINGS OF THE 36TH CHINESE CONTROL CONFERENCE (CCC 2017), 2017, : 1805 - 1810