Deep reinforcement learning based controller for ship navigation

被引：36

作者：

Deraj, Rohit ^{[1
,2
]}

Kumar, R. S. Sanjeev ^{[1
,2
]}

Alam, Md Shadab ^{[2
]}

Somayajula, Abhilash ^{[2
]}

机构：

[1] Norwegian Univ Sci & Technol NTNU, Dept Marine Technol, Trondheim, Norway

[2] Indian Inst Technol IIT Madras, Dept Ocean Engn, Chennai, India

来源：

OCEAN ENGINEERING | 2023年 / 273卷

关键词：

Reinforcement learning; Autonomous vessel; Ship maneuvering; Path-following; Deep Q-network; MMG model; Deep learning; COLLISION-AVOIDANCE; PATH;

D O I：

10.1016/j.oceaneng.2023.113937

中图分类号：

U6 [水路运输]; P75 [海洋工程];

学科分类号：

0814 ; 081505 ; 0824 ; 082401 ;

摘要：

A majority of marine accidents that occur can be attributed to errors in human decisions. Through automation, the occurrence of such incidents can be minimized. Therefore, automation in the marine industry has been receiving increased attention in the recent years. This paper investigates the automation of the path following action of a ship. A deep Q-learning approach is proposed to solve the path-following problem of a ship. This method comes under the broader area of deep reinforcement learning (DRL) and is well suited for such tasks, as it can learn to take optimal decisions through sufficient experience. This algorithm also balances the exploration and the exploitation schemes of an agent operating in an environment. A three-degree-of-freedom (3-DOF) dynamic model is adopted to describe the ship's motion. The Krisco container ship (KCS) is chosen for this study as it is a benchmark hull that is used in several studies and its hydrodynamic coefficients are readily available for numerical modeling. Numerical simulations for the turning circle and zig-zag maneuver tests are performed to verify the accuracy of the proposed dynamic model. A reinforcement learning (RL) agent is trained to interact with this numerical model to achieve waypoint tracking. Finally, the proposed approach is investigated not only by numerical simulations but also by model experiments using 1:75.5 scaled model.

引用

页数：18

共 27 条

[1]

Abadi M, 2016, PROCEEDINGS OF OSDI'16: 12TH USENIX SYMPOSIUM ON OPERATING SYSTEMS DESIGN AND IMPLEMENTATION, P265

[2]

BAKER C.C., 2005, P INT C HUM FACT SHI

[3] A knowledge-free path planning approach for smart ships based on reinforcement learning [J].

Chen, Chen ;

Chen, Xian-Qiao ;

Ma, Feng ;

Zeng, Xiao-Jun ;

Wang, Jin .

OCEAN ENGINEERING, 2019, 189

[4] Adaptive Neural Network Control of AUVs With Control Input Nonlinearities Using Reinforcement Learning [J].

Cui, Rongxin ;

Yang, Chenguang ;

Li, Yang ;

Sharma, Sanjay .

IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS, 2017, 47 (06) :1019-1029

[5]

Fossen T., 2011, Handbook of marine craft hydrodynamics and motion control

[6]

Fossen T.I., 1999, Guidance and control of ocean vehicles

[7] An Autonomous Path Planning Model for Unmanned Ships Based on Deep Reinforcement Learning [J].

Guo, Siyu ;

Zhang, Xiuguo ;

Zheng, Yisong ;

Du, Yiquan .

SENSORS, 2020, 20 (02)

[8] Risk-based implementation of COLREGs for autonomous surface vehicles using deep reinforcement learning [J].

Heiberg, Amalie ;

Larsen, Thomas Nakken ;

Meyer, Eivind ;

Rasheed, Adil ;

San, Omer ;

Varagnolo, Damiano .

NEURAL NETWORKS, 2022, 152 :17-33

[9]

Tuyen LP, 2017, 2017 IEEE INTERNATIONAL CONFERENCE ON MULTISENSOR FUSION AND INTEGRATION FOR INTELLIGENT SYSTEMS (MFI), P602, DOI 10.1109/MFI.2017.8170388

[10]

Lekkas AnastasiosM., 2012, IFAC Proceedings Volumes, V45, P398, DOI [DOI 10.3182/20120919-3-IT-2046.00068, 10.3182/20120919-3-IT-2046.00068]

← 1 2 3 →