On Robust Model-Free Reduced-Dimensional Reinforcement Learning Control for Singularly Perturbed Systems

We present a robust design for reinforcement learning (RL) based optimal control of continuous-time linear time-invariant singularly perturbed (SP) dynamic systems in the presence of dynamic uncertainties. We consider the dynamic model of both the plant and the uncertainty to be unknown. Assuming that the uncertainty satisfies an input-to-state stability (ISS) condition, we propose a variant of the adaptive dynamic programming (ADP) method that learns a sub-optimal controller using measurements of only the slow states of the plant. The resulting RL controller is, therefore, significantly reduced-dimensional, and enjoys reduced learning time. We illustrate our design with simulations of a SP system and of a clustered multi-agent consensus network.