Optimal distributed containment control for nonlinear multi-agent graphical games

A new optimal distributed control scheme is developed for the containment control problem of nonlinear multi-agent graphical games. Adaptive dynamic programming method is applied to obtain the approximate optimal control policy of each follower which make the followers converge and keep moving within the convex hull of leaders and simultaneously achieve Nash equilibrium. The performance index of each follower is defined based on local neighborhood information. Moreover, an model-free integral reinforcement learning algorithm is proposed to update the value function and the control policy without knowing the system dynamics so as to avoid to solve the coupled Hamilton-Jacobi-Bellman(HJB) equations, which is implemented by actor-critic structure and least-square method. Convergence analysis of the algorithm and asymptotical stability of the whole network are provided. Simulation results show the effectiveness of the proposed control scheme.