Differential dynamic programming for finite-horizon zero-sum differential games of nonlinear systems

In this article, we present an iterative algorithm based on differential dynamic programming (DDP) for finite-horizon two-person zero-sum differential games. The technique of DDP is used to expand the Hamilton-Jacobi-Isaacs (HJI) partial differential equation into higher-order differential equations. Using value function and saddle point approximations, the DDP expansion is transformed into algebraic matrix equation in integral form. Based on the algebraic matrix equation, a DDP iterative algorithm is developed to learn the solution to the differential games. Strict proof is proposed to guarantee the iterative convergences of the value function and saddle point. The new algorithm is fundamentally different from existing results, in the sense that it overcome the technical obstacle to address the time-varying behavior of HJI partial differential equation. Simulation examples are given to demonstrate the effectiveness of the proposed method.