Optimal Output Regulation of Partially Linear Discrete-time Systems Using Reinforcement Learning

被引：0

作者：

Pang W.-Y. ^{[1
]}

Fan J.-L. ^{[1
]}

Jiang Y. ^{[1
]}

Lewis F.L. ^{[2
]}

机构：

[1] State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University, Shenyang

[2] University of Texas at Arlington, Fort Worth

来源：

Zidonghua Xuebao/Acta Automatica Sinica | 2022年 / 48卷 / 09期

基金：

中国国家自然科学基金;

关键词：

discrete-time system; nonlinear unknown dynamics; Output regulation; reinforcement learning;

D O I：

10.16383/j.aas.c190853

中图分类号：

学科分类号：

摘要：

A data-driven control method only using online data based on reinforcement learning is proposed for the optimal output regulation problem of discrete-time partially linear systems with both linear disturbance and nonlinear uncertainties. First, the problem can be split into a constrained static optimization problem and a dynamic one. The solution of the first problem is corresponding to the solution of the regulator equation. The second can determine the optimal feedback gain of the controller. Then the small-gain theorem is used to prove the stability of the optimal output regulation problem of discrete-time partially linear systems with nonlinear uncertainties. The traditional control method needs the dynamics of the system to solve the two problems. But for this problem, a data-driven off-policy algorithm is proposed using only the measured data to find the solution of the dynamic optimization problem. Then, based on the solution of the dynamic one, the solution of the static optimization problem can be found only using data online. Finally, simulation results verify the effectiveness of the proposed method. © 2022 Science Press. All rights reserved.

引用

页码：2242 / 2253

页数：11

共 29 条

[1]

Francis B A., The linear multivariable regulator problem, SIAM Journal on Control Optimization, 15, 3, (1977)

[2]

Davison E, Goldenberg A., Robust control of a general servomechanism problem: The servo compensator, Automatica, 11, 5, (1975)

[3]

Davison E., The robust control of a servomechanism problem for linear time-invariant multivariable systems, IEEE Transactions on Automatic Control, 1, 1, (1976)

[4]

Sontag E D., Adaptation and regulation with signal detection implies internal model, System. & Control Letters, 50, 2, (2003)

[5]

Huang J., Nonlinear Output Regulation: Theory and Applications, (2004)

[6]

Saberi A, Stoorvogel A A, Sannuti P, Shi G Y., On optimal output regulation for linear systems, International Journal of Control, 76, 4, (2003)

[7]

Gao W N, Jiang Z P., Global optimal output regulation of partially linear systems via robust adaptive dynamic programming, IFAC-Papers OnLine, 48, 11, pp. 742-747, (2015)

[8]

Gao W N, Jiang Z P., Adaptive dynamics programming and adptive optimal output regulation of linear systems, IEEE Transactions on Automatic Control, 61, 12, pp. 4164-4169, (2016)

[9]

Kiumarsi B, Vamvoudakis K G, Modares H, Lewis F L., Optimal and autonomous control using reinforcement learning: A survey, IEEE Transactions on Neural Networks and Learning Systems, 29, 6, pp. 2042-2062, (2018)

[10]

Li Zhen, Fan Jia-Lu, Jiang Yi, Chai Tian-You, A model-free H∞ method based on off-policy with output data feedback, Acta Automatica Sinic, 47, 9, pp. 2182-2193, (2021)

← 1 2 3 →