Revisiting LQR Control From the Perspective of Receding-Horizon Policy Gradient

被引：4

作者：

Zhang, Xiangyuan ^{[1
]}

Basar, Tamer ^{[1
]}

机构：

[1] Univ Illinois, Dept ECE & CSL, Urbana, IL 61801 USA

来源：

IEEE CONTROL SYSTEMS LETTERS | 2023年 / 7卷

关键词：

Convergence; Complexity theory; Optimization; Optimal control; Heuristic algorithms; Computational modeling; Smoothing methods; reinforcement learning; optimization; sample complexity;

D O I：

10.1109/LCSYS.2023.3271594

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

We revisit in this letter the discrete-time linear quadratic regulator (LQR) problem from the perspective of receding-horizon policy gradient (RHPG), a newly developed model-free learning framework for control applications. We provide a fine-grained sample complexity analysis for RHPG to learn a control policy that is both stabilizing and ? -close to the optimal LQR solution, and our algorithm does not require knowing a stabilizing control policy for initialization. Combined with the recent application of RHPG in learning the Kalman filter, we demonstrate the general applicability of RHPG in linear control and estimation with streamlined analyses.

引用

页码：1664 / 1669

页数：6

共 20 条

[1]

Adams J, 2021, ADV NEUR IN, V34

[2]

Agarwal Alekh, 2020, P MACHINE LEARNING R, V125

[3]

Anderson B.D. O., 1979, Optimal Filtering, V1

[4]

Anderson BrianD. O., 1990, Optimal control: linear quadratic methods

[5]

Basar T., 1995, H INFINITY OPTIMAL C, V2nd

[6] MONOTONICITY AND STABILIZABILITY PROPERTIES OF SOLUTIONS OF THE RICCATI DIFFERENCE EQUATION - PROPOSITIONS, LEMMAS, THEOREMS, FALLACIOUS CONJECTURES AND COUNTEREXAMPLES [J].

BITMEAD, RR ;

GEVERS, MR ;

PETERSEN, IR ;

KAYE, RJ .

SYSTEMS & CONTROL LETTERS, 1985, 5 (05) :309-315

[7]

Bu JJ, 2019, Arxiv, DOI arXiv:1907.08921

[8]

Fazel M, 2018, PR MACH LEARN RES, V80

[9] POLICY GRADIENT METHODS FOR THE NOISY LINEAR QUADRATIC REGULATOR OVER A FINITE HORIZON [J].

Hambly, Ben ;

Xu, Renyuan ;

Yang, Huining .

SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 2021, 59 (05) :3359-3391

[10]

Hassibi B., 1999, INDEFINITE QUADRATIC

← 1 2 →