Model-free finite-horizon optimal control of discrete-time two-player zero-sum games

被引：3

作者：

Wang, Wei ^{[1
,2
,3
]}

Chen, Xin ^{[2
,3
,4
]}

Du, Jianhua ^{[1
]}

机构：

[1] Zhongnan Univ Econ & Law, Sch Informat & Safety Engn, Wuhan, Peoples R China

[2] Hubei Key Lab Adv Control & Intelligent Automat C, Wuhan, Peoples R China

[3] Minist Educ, Engn Res Ctr Intelligent Technol Geoexplorat, Wuhan, Peoples R China

[4] China Univ Geosci, Sch Automat, Wuhan, Peoples R China

来源：

INTERNATIONAL JOURNAL OF SYSTEMS SCIENCE | 2023年 / 54卷 / 01期

关键词：

Q-function; finite horizon; optimal control; zero-sum game; H-INFINITY-CONTROL; DIFFERENTIAL-GAMES; SYSTEMS;

D O I：

10.1080/00207721.2022.2111236

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Conventionally, as the system dynamics is known, the finite-horizon optimal control of zero-sum games relies on solving the time-varying Riccati equations. In this paper, with unknown system dynamics being considered, a Q-function-based finite-horizon control method is introduced to approximate the solutions of the time-varying Riccati equations. First, a time-varying Q-function explicitly dependent on the time-varying control and disturbance is defined. Then the defined time-varying Q-function is utilised to represent the time-varying control and disturbance which are equivalent to the solutions of the time-varying Riccati equations by relaxing the system dynamics. Finally, a model-free method is introduced to approximate the defined time-varying Q-function. Simulation studies are conducted to demonstrate the validity of the developed method.

引用

页码：167 / 179

页数：13

共 50 条

[1] Sufficient Conditions for Optimality in Finite-Horizon Two-Player Zero-Sum Hybrid Games
Leudo, Santiago J.
Sanfelice, Ricardo G.
2022 IEEE 61ST CONFERENCE ON DECISION AND CONTROL (CDC), 2022, : 3268 - 3273
[2] Finite Horizon Stochastic Optimal Control of Nonlinear Two-Player Zero-Sum Games under Communication Constraint
Xu, Hao
Jagannathan, S.
PROCEEDINGS OF THE 2014 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2014, : 239 - 244
[3] Model-free finite-horizon optimal tracking control of discrete-time linear systems
Wang, Wei
Xie, Xiangpeng
Feng, Changyang
APPLIED MATHEMATICS AND COMPUTATION, 2022, 433
[4] TWO-PLAYER ZERO-SUM STOCHASTIC DIFFERENTIAL GAMES WITH RANDOM HORIZON
Ferreira, M.
Pinheiro, D.
Pinheiro, S.
ADVANCES IN APPLIED PROBABILITY, 2019, 51 (04) : 1209 - 1235
[5] Finite-horizon Q-learning for discrete-time zero-sum games with application to H∞$$ {H}_{\infty } $$ control
Liu, Mingxiang
Cai, Qianqian
Meng, Wei
Li, Dandan
Fu, Minyue
ASIAN JOURNAL OF CONTROL, 2023, 25 (04) : 3160 - 3168
[6] Finite-Horizon Near Optimal Design of Nonlinear Two-Player Zero-Sum Game in Presence of Completely Unknown Dynamics
School of Engineering and Computing Sciences, Texas A&M University-Corpus Christi, Corpus Christi
TX
78414, United States
J. Control Autom. Electr. Syst., 4 (361-370):
[7] Output feedback Q-learning for discrete-time finite-horizon zero-sum games with application to the H? control
Liu, Mingxiang
Cai, Qianqian
Li, Dandan
Meng, Wei
Fu, Minyue
NEUROCOMPUTING, 2023, 529 : 48 - 55
[8] Finite-Horizon Near Optimal Design of Nonlinear Two-Player Zero-Sum Game in Presence of Completely Unknown Dynamics
Xu, Hao
JOURNAL OF CONTROL AUTOMATION AND ELECTRICAL SYSTEMS, 2015, 26 (04) : 361 - 370
[9] Model-Free Learning for Two-Player Zero-Sum Partially Observable Markov Games with Perfect Recall
Kozuno, Tadashi
Menard, Pierre
Munos, Remi
Valko, Michal
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 34 (NEURIPS 2021), 2021, 34
[10] A PROBABILISTIC VERIFICATION THEOREM FOR THE FINITE HORIZON TWO-PLAYER ZERO-SUM OPTIMAL SWITCHING GAME IN CONTINUOUS TIME
Hamadene, S.
Martyr, R.
Moriarty, J.
ADVANCES IN APPLIED PROBABILITY, 2019, 51 (02) : 425 - 442

← 1 2 3 4 5 →