A Single-NN Iterative Adaptive Dynamic Programming Algorithm for Continuous-Time Nonlinear Zero-Sum Games

This paper establishes an approximate optimal critic learning algorithm based on single-network adaptive dynamic programming (ADP) aiming at solving for continuous-time 2-player zero-sum games(ZSG). However, the situation where the accurate dynamics is influenced by disturbance will occur from time to time. Because neural network(NN) is used in this paper, we have to face the approximation error, which will disturb the control. In order to surmount this problem, we use online data to calculate the weights of NN, and design robust controller to stabilize the disturbed nonlinear system. In other way, we used policy iteration and integral reinforcement learning to settle the Hamilton-Jacobi-Isaacs equation. And through the least-squares method, the NN weights are solved. Based on the theoretical analysis, this algorithm is a derivation from Gauss-Newton method, which can solve an optimization problem without disturbance. Thus it will converge to the optimal value. Because large quantities of online data are used, the process will accurately converge optimal control. Simulation results can verify that it's realizable to deal with disturbed nonlinear ZSG.