Distributed cooperative H∞ optimal control of underactuated autonomous underwater vehicles based on reinforcement learning and prescribed performance

To balance energy resources and control performance, an H-infinity optimal control method based on prescribed performance control (PPC) and a reinforcement learning (RL) algorithm with actor-critic mechanisms for distributed cooperative control is proposed for multiple five-degree-of-freedom underactuated autonomous underwater vehicles (AUVs) with unknown uncertainty disturbances. First, an optimal control strategy combining PPC is proposed to achieve optimal control of a cooperative system while ensuring that the error always stays within the prescribed boundary. Second, to suppress uncertainty disturbances, H-infinity control methods are proposed to improve the robustness of the system. Achieving H-infinity optimal control requires solving the Hamilton-Jacobi-Bellman (HJB) equation, but the inherent nonlinearity of the HJB equation makes it difficult to solve. Therefore, an adaptive approximation strategy incorporating an online RL method with an actor-critic architecture is used to solve the above problem, which dynamically adjusts the control strategy to ensure system control performance through the environment assessment-feedback approach. In addition, a distributed adaptive state observer is proposed to obtain information about the virtual leader for each agent so that leader information can be accurately obtained, even if the agent communicates only with neighboring agents. Using the above control method, all errors of the formation system are proven to be uniform and ultimately bounded according to Lyapunov's stability theorem. Finally, a numerical simulation is performed to further demonstrate the effectiveness and feasibility of the proposed method.