Knowledge transfer enabled reinforcement learning for efficient and safe autonomous ship collision avoidance

Research on collision avoidance decision-making (CADM) for autonomous ships is a very challenging task in the shipping field. Considered one of the machine learning algorithms that has received considerable attention, reinforcement learning technology enables actions to be continually optimized by agents interacting with the environment, aiming to maximize rewards and returns. Significant potential is attributed to the research on autonomous ship collision avoidance. To investigate an efficient and practical ship collision avoidance algorithm, the knowledge transfer (KT) method is employed in this research to introduce an improved reinforcement learning approach. With a thorough understanding of ship collision avoidance behavior and the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs), a reward function is designed to guide and constrain ship collision avoidance behavior. Subsequently, ship collision avoidance tasks are categorized, and knowledge from source tasks is extracted and transferred to closely related target tasks. Experiments have been conducted across various collision avoidance tasks, encompassing diverse types and degrees of similarity. In multi-ships cases, the success rate of the learned knowledge applications of head-on, overtaking, and crossing encounter cases are 90%, 95%, and 82.5% respectively. The outcomes demonstrate that the proposed method enhances algorithmic efficiency while satisfying the requirements for safety and rule compliance in ship collision avoidance behavior. Furthermore, the methodology could also benefit other autonomous systems in dynamic environments.