Stochastic Optimal Control of Unknown Linear Networked Control System using Q-Learning Methodology

被引：0

作者：

Xu, Hao ^{[1
]}

Jagannathan, S. ^{[1
]}

机构：

[1] Missouri Univ Sci & Technol, Dept Elect & Comp Engn, Rolla, MO 65409 USA

来源：

2011 AMERICAN CONTROL CONFERENCE | 2011年

关键词：

Networked Control System (NCS); Q-function; Adaptive Estimator (AE); Optimal Control;

D O I：

暂无

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

In this paper, the Bellman equation is utilized forward-in-time for the stochastic optimal control of Networked Control System (NCS) with unknown system dynamics in the presence of random delays and packet losses which are unknown. The proposed stochastic optimal control approach, referred normally as adaptive dynamic programming, uses an adaptive estimator (AE) and ideas from Q-learning to solve the infinite horizon optimal regulation control of NCS with unknown system dynamics. Update laws for tuning the unknown parameters of the adaptive estimator (AE) online to obtain the time-based Q-function are derived. Lyapunov theory is used to show that all signals are asymptotically stable (AS) and that the approximated control signals converge to optimal control inputs. Simulation results are included to show the effectiveness of the proposed scheme.

引用

页码：2819 / 2824

页数：6

共 50 条

[21] Quantized measurements in Q-learning based model-free optimal control [J].

Tiistola, Sini ;

Ritala, Risto ;

Vilkko, Matti .

IFAC PAPERSONLINE, 2020, 53 (02) :1640-1645

[22] Optimal Control for A Class of Linear Stochastic Impulsive Systems with Partially Unknown Information [J].

Wu, Yan ;

Luo, Shixian .

2023 35TH CHINESE CONTROL AND DECISION CONFERENCE, CCDC, 2023, :1768-1773

[23] Discounted linear Q-learning control with novel tracking cost and its stability [J].

Wang, Ding ;

Ren, Jin ;

Ha, Mingming .

INFORMATION SCIENCES, 2023, 626 :339-353

[24] Optimal Tracking Current Control of Switched Reluctance Motor Drives Using Reinforcement Q-Learning Scheduling [J].

Alharkan, Hamad ;

Saadatmand, Sepehr ;

Ferdowsi, Mehdi ;

Shamsi, Pourya .

IEEE ACCESS, 2021, 9 :9926-9936

[25] On the effect of probing noise in optimal control LQR via Q-learning using adaptive filtering algorithms [J].

Lopez Yanez, Williams Jesus ;

de Souza, Francisco das Chagas .

EUROPEAN JOURNAL OF CONTROL, 2022, 65

[26] Using Q-learning and genetic algorithms to improve the efficiency of weight adjustments for optimal control and design problems [J].

Kamali, Kaivan ;

Jiang, L. J. ;

Yen, John ;

Wang, K. W. .

JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING, 2007, 7 (04) :302-308

[27] Optimal Control for Interconnected Multi-Area Power Systems With Unknown Dynamics: An Off-Policy Q-Learning Method [J].

Wang, Jing ;

Mi, Xuanrui ;

Shen, Hao ;

Park, Ju H. ;

Shi, Kaibo .

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS, 2024, 71 (05) :2849-2853

[28] Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach [J].

Vamvoudakis, Kyriakos G. .

SYSTEMS & CONTROL LETTERS, 2017, 100 :14-20

[29] Model-free optimal control for discrete-time Markovian jump linear systems: A Q-learning approach [J].

Badfar, Ehsan ;

Tavassoli, Babak .

JOURNAL OF THE FRANKLIN INSTITUTE, 2025, 362 (12)

[30] An Optimal Hybrid Learning Approach for Attack Detection in Linear Networked Control Systems [J].

Haifeng Niu ;

Avimanyu Sahoo ;

Chandreyee Bhowmick ;

SJagannathan .

IEEE/CAA Journal of Automatica Sinica, 2019, 6 (06) :1404-1416

← 1 2 3 4 5 →