Model-free optimal tracking control for linear discrete-time stochastic systems subject to additive and multiplicative noises

被引：0

作者：

Yin Y.-B. ^{[1
]}

Luo S.-X. ^{[1
]}

Wan T. ^{[2
]}

机构：

[1] School of Electrical Engineering, Guangxi University, Guangxi, Nanning

[2] School of Automation Science and Technology, South China University of Technology, Guangdong, Guangzhou

来源：

Kongzhi Lilun Yu Yingyong/Control Theory and Applications | 2023年 / 40卷 / 06期

基金：

中国国家自然科学基金;

关键词：

model-free control; Q-learning; stochastic linear quadratic tracking; stochastic noise;

D O I：

10.7641/CTA.2022.11105

中图分类号：

学科分类号：

摘要：

This paper focuses on the optimal tracking control problem for a class of discrete-time stochastic systems with additive and multiplicative noises. By constructing an augmented system composed of the original system and the reference generator, the cost function of the stochastic linear quadratic tracking control (SLQT) is transformed into a quadratic function in terms of the augmented state. A Bellman equation and an augmented stochastic algebraic Riccati equation (SARE) for solving the SLQT are then derived. For the case of both the system and reference generator dynamics are completely unknown, a Q-learning algorithm is proposed to solve the augmented SARE, and its convergence is proved rigorously. Moreover, the online model-free control algorithm implements with the batch least square method (BLS). The effectiveness of the proposed control scheme is verified by the single-phase voltage-source UPS inverter. © 2023 South China University of Technology. All rights reserved.

引用

页码：1014 / 1022

页数：8

共 21 条

[1]

LEWIS F L, VRABIE D L, SYRMOS V L., Optimal Control, (2012)

[2]

BERTSEKAS D P., Reinforcement Learning and Optimal Control, (2019)

[3]

MODARES H, LEWIS F L., Linear quadratic tracking control of partially-unknown continuous-time systems using reinforcement learning, IEEE Transactions on Automatic Control, 59, 11, pp. 3051-3056, (2014)

[4]

KIUMARSI B, LEWIS F L, MODARES H, Et al., Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics, Automatica, 50, 4, pp. 1167-1175, (2014)

[5]

SUN W J, ZHAO G Y, PENG Y J., Adaptive optimal output feedback tracking control for unknown discrete-time linear systems using a combined reinforcement Q-learning and internal model method, IET Control Theory & Applications, 13, 18, pp. 3075-3086, (2019)

[6]

BIAN T, JIANG Z P., Reinforcement learning and adaptive optimal control for continuous-time nonlinear systems: A value iteration approach, IEEE Transactions on Neural Networks and Learning Systems, 33, 7, pp. 2781-2790, (2021)

[7]

RAJASEKARAN P K, SATYANARAYANA N, SRINATH M D., Optimum linear estimation of stochastic signals in the presence of multiplicative noise, IEEE Transactions on Aerospace and Electronic Systems, 7, 3, pp. 462-468, (1971)

[8]

DING D, WANG Z, WEI G, Et al., Event-based security control for discrete-time stochastic systems, IET Control Theory & Applications, 10, 15, pp. 1808-1815, (2016)

[9]

GUAN Z H, CHEN C Y, FENG G, Et al., Optimal tracking performance limitation of networked control systems with limited bandwidth and additive colored white gaussian noise, IEEE Transactions on Circuits & Systems I Regular Papers, 60, 1, pp. 189-198, (2013)

[10]

WANG T, ZHANG H G, LUO Y H., Stochastic linear quadratic optimal control for model-free discrete-time systems based on Q-learning algorithm, Neurocomputing, 312, 27, pp. 1-8, (2018)

← 1 2 3 →