Stochastic Optimal Control of Unknown Linear Networked Control System using Q-Learning Methodology

被引：0
作者：
Xu, Hao [1 ]
Jagannathan, S. [1 ]
机构：
[1] Missouri Univ Sci & Technol, Dept Elect & Comp Engn, Rolla, MO 65409 USA
来源：
2011 AMERICAN CONTROL CONFERENCE | 2011年
关键词：
Networked Control System (NCS); Q-function; Adaptive Estimator (AE); Optimal Control;
D O I：
暂无
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
In this paper, the Bellman equation is utilized forward-in-time for the stochastic optimal control of Networked Control System (NCS) with unknown system dynamics in the presence of random delays and packet losses which are unknown. The proposed stochastic optimal control approach, referred normally as adaptive dynamic programming, uses an adaptive estimator (AE) and ideas from Q-learning to solve the infinite horizon optimal regulation control of NCS with unknown system dynamics. Update laws for tuning the unknown parameters of the adaptive estimator (AE) online to obtain the time-based Q-function are derived. Lyapunov theory is used to show that all signals are asymptotically stable (AS) and that the approximated control signals converge to optimal control inputs. Simulation results are included to show the effectiveness of the proposed scheme.
引用
收藏
页码：2819 / 2824
页数：6
相关论文
共 50 条
  • [11] Reinforcement Q-Learning Algorithm for H∞ Tracking Control of Unknown Discrete-Time Linear Systems
    Peng, Yunjian
    Chen, Qian
    Sun, Weijie
    IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS, 2020, 50 (11): : 4109 - 4122
  • [12] Adaptive optimal output feedback tracking control for unknown discrete-time linear systems using a combined reinforcement Q-learning and internal model method
    Sun, Weijie
    Zhao, Guangyue
    Peng, Yunjian
    IET CONTROL THEORY AND APPLICATIONS, 2019, 13 (18) : 3075 - 3086
  • [13] Optimal Tracking Control of Nonlinear Multiagent Systems Using Internal Reinforce Q-Learning
    Peng, Zhinan
    Luo, Rui
    Hu, Jiangping
    Shi, Kaibo
    Nguang, Sing Kiong
    Ghosh, Bijoy Kumar
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022, 33 (08) : 4043 - 4055
  • [14] Reinforcement Q-Learning Incorporated With Internal Model Method for Output Feedback Tracking Control of Unknown Linear Systems
    Chen, Cong
    Sun, Weijie
    Zhao, Guangyue
    Peng, Yunjian
    IEEE ACCESS, 2020, 8 : 134456 - 134467
  • [15] Stochastic optimal control of networked control systems with control packet dropouts
    Shouwan Gao
    Gongyou Tang
    Journal of Control Theory and Applications, 2012, 10 (3): : 410 - 414
  • [16] Adaptive traffic signal control using deep Q-learning: case study on optimal implementations
    Pan, Guangyuan
    Muresan, Matthew
    Fu, Liping
    CANADIAN JOURNAL OF CIVIL ENGINEERING, 2023, 50 (06) : 488 - 497
  • [17] Stochastic optimal control of networked control systems with control packet dropouts
    Shouwan GAO~1
    2.College of Information Science and Engineering
    Control Theory and Technology, 2012, 10 (03) : 410 - 414
  • [18] Optimal Control with Learning on the Fly: System with Unknown Drift
    Gurevich, Daniel
    Goswami, Debdipta
    Fefferman, Charles L.
    Rowley, Clarence W.
    LEARNING FOR DYNAMICS AND CONTROL CONFERENCE, VOL 168, 2022, 168
  • [19] Discrete-Time Optimal Control Scheme Based on Q-Learning Algorithm
    Wei, Qinglai
    Liu, Derong
    Song, Ruizhuo
    2016 SEVENTH INTERNATIONAL CONFERENCE ON INTELLIGENT CONTROL AND INFORMATION PROCESSING (ICICIP), 2016, : 125 - 130
  • [20] Quantized measurements in Q-learning based model-free optimal control
    Tiistola, Sini
    Ritala, Risto
    Vilkko, Matti
    IFAC PAPERSONLINE, 2020, 53 (02): : 1640 - 1645
12345