Trajectory tracking control of vectored thruster autonomous underwater vehicles based on deep reinforcement learning

Vectored thruster autonomous underwater vehicles (AUVs) offer superior manoeuvrability compared to traditional fin and rudder systems, especially at low or zero velocities. However, controlling these AUVs becomes challenging in the presence of unpredictable interference forces and external water flow velocities. This paper introduces a novel three-dimensional trajectory tracking method for vectored thruster AUVs using reinforcement learning, enabling the system to learn optimal control policies through environmental interaction. A 5-degree of freedom (5-DOF) model is developed from the kinematics and dynamics equations of the underactuated AUV. An extended state observer (ESO) is used to estimate the differential expansion state based on observed variables. A reinforcement learning-based trajectory tracking strategy is then implemented. Simulation experiments demonstrate the method's effectiveness in precisely controlling AUVs under varying environmental conditions, achieving exceptional tracking accuracy even in the presence of random interference forces and external water flow velocities.