Game-based distributed optimal formation tracking control of underactuated AUVs based on reinforcement learning

This paper investigates game-based distributed optimal time-varying formation tracking control problem based on backstepping technique and reinforcement learning (RL) for autonomous underwater vehicles (AUVs) with time-varying disturbances and input saturation. The proposed method enables the multi-AUV to track a desired trajectory while holding the time-varying formation. To accomplish this, we adopt a game between leader and followers to enhance robustness of the system to time-varying disturbances. The approximate optimal solution of the established Hamilton-Jacobi-Isaacs (HJI) equation is obtained by RL which performs online via the single critic Neural Network (SCNN). Hence, the optimal solution corresponds to the saddle point equilibrium of the tracking game. In addition, the command filter is used to construct the unknown hydrodynamic parameter and time-varying disturbances. Furthermore, we introduce a quasi-norm to confront input saturation. The effectiveness of the proposed approach is verified by the simulation results provided.