Q-learning based tracking control with novel finite-horizon performance index ☆

被引：0
作者：
Wang, Wei [1 ,2 ,3 ]
Wang, Ke [1 ]
Huang, Zixin [4 ]
Mu, Chaoxu [1 ]
Shi, Haoxian [5 ]
机构：
[1] Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China
[2] Zhongnan Univ Econ & Law, Sch Informat Engn, Wuhan 430073, Peoples R China
[3] Zhongnan Univ Econ & Law, Emergency Management Res Ctr, Wuhan 430073, Peoples R China
[4] Wuhan Inst Technol, Sch Elect & Informat Engn, Wuhan 430205, Peoples R China
[5] China Geol Survey, Guangzhou Marine Geol Survey, Guangzhou 510075, Peoples R China
来源：
INFORMATION SCIENCES | 2024年 / 681卷
关键词：
Optimal tracking control; Model-free control; Q-function; Finite-horizon; NONLINEAR-SYSTEMS; TIME-SYSTEMS;
D O I：
10.1016/j.ins.2024.121212
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
A data-driven method is designed to realize the model-free finite-horizon optimal tracking control (FHOTC) of unknown linear discrete-time systems based on Q-learning in this paper. First, a novel finite-horizon performance index (FHPI) that only depends on the next-step tracking error is introduced. Then, an augmented system is formulated, which incorporates with the system model and the trajectory model. Based on the novel FHPI, a derivation of the augmented time-varying Riccati equation (ATVRE) is provided. We present a data-driven FHOTC method that uses Qlearning to optimize the defined time-varying Q-function. This allows us to estimate the solutions of the ATVRE without the system dynamics. Finally, the validity and features of the proposed Qlearning-based FHOTC method are demonstrated by means of conducting comparative simulation studies.
引用
收藏
页数：10
相关论文
共 50 条
  • [11] Finite-Horizon H∞ Tracking Control for Unknown Nonlinear Systems With Saturating Actuators
    Zhang, Huaguang
    Cui, Xiaohong
    Luo, Yanhong
    Jiang, He
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2018, 29 (04) : 1200 - 1212
  • [12] Reinforcement Learning for Finite-Horizon H∞ Tracking Control of Unknown Discrete Linear Time-Varying System
    Ye, Linwei
    Zhao, Zhonggai
    Liu, Fei
    IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS, 2024, 54 (10): : 6385 - 6396
  • [13] Model-free finite-horizon optimal tracking control of discrete-time linear systems
    Wang, Wei
    Xie, Xiangpeng
    Feng, Changyang
    APPLIED MATHEMATICS AND COMPUTATION, 2022, 433
  • [14] A Tractable Algorithm For Finite-Horizon Continuous Reinforcement Learning
    Gampa, Phanideep
    Kondamudi, Sairam Satwik
    Lakshmanan, K.
    2019 2ND INTERNATIONAL CONFERENCE ON INTELLIGENT AUTONOMOUS SYSTEMS (ICOIAS 2019), 2019, : 63 - 69
  • [15] Finite-horizon optimal control for continuous-time uncertain nonlinear systems using reinforcement learning
    Zhao, Jingang
    Gan, Minggang
    INTERNATIONAL JOURNAL OF SYSTEMS SCIENCE, 2020, 51 (13) : 2429 - 2440
  • [16] Optimal Tracking Control of Servo Motor Speed Based on Online Supplementary Q-Learning
    Zou X.
    Xiao X.
    He Q.
    Vyacheslav S.
    Diangong Jishu Xuebao/Transactions of China Electrotechnical Society, 2019, 34 (05): : 917 - 923
  • [17] Approximate finite-horizon optimal control without PDEs
    Sassano, M.
    Astolfi, A.
    SYSTEMS & CONTROL LETTERS, 2013, 62 (02) : 97 - 103
  • [18] FINITE-HORIZON ε-OPTIMAL TRACKING CONTROL OF DISCRETE-TIME LINEAR SYSTEMS USING ITERATIVE APPROXIMATE DYNAMIC PROGRAMMING
    Tan, Fuxiao
    Luo, Bin
    Guan, Xinping
    ASIAN JOURNAL OF CONTROL, 2015, 17 (01) : 176 - 189
  • [19] Finite-horizon optimal control for unknown systems with saturating control inputs
    Cui X.-H.
    Luo Y.-H.
    Zhang H.-G.
    Zu P.-F.
    Zhang, Hua-Guang (hgzhang@ieee.org), 2016, South China University of Technology (33): : 631 - 637
  • [20] Switching control of morphing aircraft based on Q-learning
    Gong, Ligang
    Wang, Qing
    Hu, Changhua
    Liu, Chen
    CHINESE JOURNAL OF AERONAUTICS, 2020, 33 (02) : 672 - 687
12345