Online Learning-based Optimal Control of Nonlinear Systems with Finite-Time Convergence Guarantees

This paper develops a critic-only reinforcement learning-based algorithm for learning the solution to the Hamilton-Jacobi-Bellman equation in finite time. In particular, a non-Lipschitz experience replay-based learning law utilizing recorded and current data is introduced for updating the critic weights to learn the value function. The non-Lipschitz property of the dynamics gives rise to finite-time convergence and stability, while the experience replay-based approach eliminates the need to satisfy the persistence of excitation condition if the recorded data is sufficiently rich. Simulation results demonstrate the efficacy of the proposed approach.