Data-based optimal control design with reinforcement learning for nonlinear PDE systems

In this paper, we investigate the optimal controller design for partial differential equation (PDE) systems which can't construct the accurate mathematical models. Firstly, the empirical eigenfunctions (EEFs) of the PDE system are calculated by the Karhunen-Loeve decomposition. The empirical eigenfunctions are used to transform the PDE into the high-order ordinary differential equation (ODE). Then, using the singular perturbation theorem, the above mentioned ODE system is approximated as a reduced-order model. Furthermore, we reconstruct the Hamilton-Jacobi-Bellman (HJB) equation according to an optimization performance index of the reduced-order ODE model. A policy iteration scheme relied on a reinforcement learning approach is proposed to calculate the HJB equation online. Finally, experimental simulations are carried out to demonstrate the effectiveness and the practicability of the proposed control method.