Collision avoidance for an unmanned surface vehicle using deep reinforcement learning

In this paper, a deep reinforcement learning (DRL)-based collision avoidance method is proposed for an unmanned surface vehicle (USV). This approach is applicable to the decision-making stage of collision avoidance, which determines whether the avoidance is necessary, and if so, determines the direction of the avoidance maneuver. To utilize the visual recognition capability of deep neural networks as a tool for analyzing the complex and ambiguous situations that are typically encountered, a grid map representation of the ship encounter situation was suggested. For the composition of the DRL network, we proposed a neural network architecture and semi-Markov decision process model that was specially designed for the USV collision avoidance problem. The proposed DRL network was trained through repeated simulations of collision avoidance. After the training process, the DRL network was implemented in collision avoidance experiments and simulations to evaluate its situation recognition and collision avoidance capability.