Robust value iteration for optimal control of discrete-time linear systems

This paper investigates properties of value iteration in the presence of deviations, starting from a benchmark control problem for discrete-time linear systems. Using properties of invariant metrics, value iteration for the considered control problem is demonstrated to be robust to small deviations. Specifically, value iteration enjoys a non-asymptotic convergence property when the deviations keep small in the execution, and generates solutions that converge to a small neighborhood of the optimal ones. As an extension, an optimistic model-free value iteration is proposed for systems suffering from additive noise of zero mean with the estimation error analysis and convergence analysis. The proposed results are illustrated through numerical simulations. (c) 2025 Published by Elsevier Ltd.