Autonomous underwater vehicle path planning based on actor-multi-critic reinforcement learning

被引:16
|
作者
Wang, Zhuo [1 ,2 ]
Zhang, Shiwei [1 ]
Feng, Xiaoning [3 ]
Sui, Yancheng [1 ]
机构
[1] Harbin Engn Univ, Sci & Technol Underwater Vehicle Lab, Harbin, Peoples R China
[2] Peng Cheng Lab, Shenzhen, Peoples R China
[3] Harbin Engn Univ, Coll Comp Sci & Technol, Harbin 150001, Peoples R China
基金
中国国家自然科学基金;
关键词
Autonomous underwater vehicle; path planning; dynamic obstacle avoidance; actor-critic; neural networks; FEEDFORWARD NETWORKS; ENVIRONMENT;
D O I
10.1177/0959651820937085
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The environmental adaptability of autonomous underwater vehicles is always a problem for its path planning. Although reinforcement learning can improve the environmental adaptability, the slow convergence of reinforcement learning is caused by multi-behavior coupling, so it is difficult for autonomous underwater vehicle to avoid moving obstacles. This article proposes a multi-behavior critic reinforcement learning algorithm applied to autonomous underwater vehicle path planning to overcome problems associated with oscillating amplitudes and low learning efficiency in the early stages of training which are common in traditional actor-critic algorithms. Behavior critic reinforcement learning assesses the actions of the actor from perspectives such as energy saving and security, combining these aspects into a whole evaluation of the actor. In this article, the policy gradient method is selected as the actor part, and the value function method is selected as the critic part. The strategy gradient and the value function methods for actor and critic, respectively, are approximated by a backpropagation neural network, the parameters of which are updated using the gradient descent method. The simulation results show that the method has the ability of optimizing learning in the environment and can improve learning efficiency, which meets the needs of real time and adaptability for autonomous underwater vehicle dynamic obstacle avoidance.
引用
收藏
页码:1787 / 1796
页数:10
相关论文
共 50 条
  • [31] End-to-end autonomous underwater vehicle path following control method based on improved soft actor-critic for deep space exploration
    Dong, Na
    Liu, Shoufu
    Ip, Andrew W. H.
    Yung, Kai Leung
    Gao, Zhongke
    Juan, Rongshun
    Wang, Yanhui
    JOURNAL OF INDUSTRIAL INFORMATION INTEGRATION, 2025, 45
  • [32] 3-Dimensional Path Planning for Autonomous Underwater Vehicle
    Wang, Yiheng
    Zheng, Yahong Rosa
    OCEANS 2018 MTS/IEEE CHARLESTON, 2018,
  • [33] Multiple Region Coverage Path Planning for Autonomous Underwater Vehicle
    Sinha, Shristi Deva
    DEFENCE SCIENCE JOURNAL, 2021, 71 (06) : 784 - 790
  • [34] A quick algorithm for planning a path for a biomimetic autonomous underwater vehicle
    Praczyk, Tomasz
    SCIENTIFIC JOURNALS OF THE MARITIME UNIVERSITY OF SZCZECIN-ZESZYTY NAUKOWE AKADEMII MORSKIEJ W SZCZECINIE, 2016, 45 (117): : 23 - 28
  • [35] Dynamic Target Tracking of Autonomous Underwater Vehicle Based on Deep Reinforcement Learning
    Shi, Jiaxiang
    Fang, Jianer
    Zhang, Qizhong
    Wu, Qiuxuan
    Zhang, Botao
    Gao, Farong
    JOURNAL OF MARINE SCIENCE AND ENGINEERING, 2022, 10 (10)
  • [36] Path Planning for Autonomous Underwater Vehicle in Presence of Moving Obstacle Based on Three Inputs Fuzzy Logic
    Li, Xiangjun
    Wang, Wei
    Song, Jinyan
    Liu, Dan
    2019 4TH ASIA-PACIFIC CONFERENCE ON INTELLIGENT ROBOT SYSTEMS (ACIRS 2019), 2019, : 265 - 268
  • [37] An evaluation of path-planning methods for autonomous underwater vehicle based on terrain-aided navigation
    Cong, Zheng
    Li, Ye
    Jiang, Yanqing
    Ma, Teng
    Gong, Yusen
    Wang, Rupeng
    Wu, Haowei
    INTERNATIONAL JOURNAL OF ADVANCED ROBOTIC SYSTEMS, 2019, 16 (03):
  • [38] Path planning for autonomous underwater vehicle based on an enhanced water wave optimization algorithm
    Yan, Zheping
    Zhang, Jinzhong
    Tang, Jialing
    MATHEMATICS AND COMPUTERS IN SIMULATION, 2021, 181 : 192 - 241
  • [39] Path Planning for Autonomous Underwater Vehicle Based on Artificial Potential Field and Modified RRT
    Zhu, Jia
    Zhao, Shili
    Zhao, Ran
    2021 INTERNATIONAL CONFERENCE ON COMPUTER, CONTROL AND ROBOTICS (ICCCR 2021), 2021, : 21 - 25
  • [40] Path Planning for Autonomous Balloon Navigation with Reinforcement Learning
    He, Yingzhe
    Guo, Kai
    Wang, Chisheng
    Fu, Keyi
    Zheng, Jiehao
    ELECTRONICS, 2025, 14 (01):