Dynamic robotic tracking of underwater targets using reinforcement learning

被引:22
作者
Masmitja, I. [1 ,2 ]
Martin, M. [3 ,4 ]
O'Reilly, T. [2 ]
Kieft, B. [2 ]
Palomeras, N. [5 ]
Navarro, J. [1 ]
Katija, K. [2 ]
机构
[1] CSIC, Inst Ciencies Mar ICM, Barcelona 95062, Spain
[2] Monterey Bay Aquarium Res Inst MBARI, Bioinspirat Lab, Res & Dev, Moss Landing, CA 95062 USA
[3] Univ Politecn Cataluna, Knowledge Engn & Machine Learning Grp, Barcelona Tech, Barcelona 08034, Spain
[4] Barcelona Supercomp Ctr BSC, Barcelona 08034, Spain
[5] Univ Girona, Comp Vis & Robot Inst, Girona 17003, Spain
关键词
LEVEL CONTROL; NAVIGATION; NETWORKS; VEHICLE; AUV; LOCALIZATION;
D O I
10.1126/scirobotics.ade7811
中图分类号
TP24 [机器人技术];
学科分类号
080202 ; 1405 ;
摘要
To realize the potential of autonomous underwater robots that scale up our observational capacity in the ocean, new techniques are needed. Fleets of autonomous robots could be used to study complex marine systems and animals with either new imaging configurations or by tracking tagged animals to study their behavior. These activities can then inform and create new policies for community conservation. The role of animal connectivity via active movement of animals represents a major knowledge gap related to the distribution of deep ocean populations. Tracking underwater targets represents a major challenge for observing biological processes in situ, and methods to robustly respond to a changing environment during monitoring missions are needed. Analytical techniques for optimal sensor placement and path planning to locate underwater targets are not straightforward in such cases. The aim of this study was to investigate the use of reinforcement learning as a tool for range-only underwater target-tracking optimization, whose promising capabilities have been demonstrated in terrestrial scenarios. To evaluate its usefulness, a reinforcement learning method was implemented as a path planning system for an autonomous surface vehicle while tracking an underwater mobile target. A complete description of an open-source model, performance metrics in simulated environments, and evaluated algorithms based on more than 15 hours of at-sea field experiments are presented. These efforts demonstrate that deep reinforcement learning is a powerful approach that enhances the abilities of autonomous robots in the ocean and encourages the deployment of algorithms like these for monitoring marine biological systems in the future.
引用
收藏
页数:11
相关论文
共 65 条
[1]   The generation of plankton patchiness by turbulent stirring [J].
Abraham, ER .
NATURE, 1998, 391 (6667) :577-580
[2]  
Agarwal Rishabh, 2021, ADV NEURAL INFORM PR, V34
[3]   Docking Control of an Autonomous Underwater Vehicle Using Reinforcement Learning [J].
Anderlini, Enrico ;
Parker, Gordon G. ;
Thomas, Giles .
APPLIED SCIENCES-BASEL, 2019, 9 (17)
[4]  
[Anonymous], 1998, Introduction to reinforcement learning
[5]   Personalized digital marketing recommender engine [J].
Behera, Rajat Kumar ;
Gunasekaran, Angappa ;
Gupta, Shivam ;
Kamboj, Shampy ;
Bala, Pradip Kumar .
JOURNAL OF RETAILING AND CONSUMER SERVICES, 2020, 53
[6]   Autonomous navigation of stratospheric balloons using reinforcement learning [J].
Bellemare, Marc G. ;
Candido, Salvatore ;
Castro, Pablo Samuel ;
Gong, Jun ;
Machado, Marlos C. ;
Moitra, Subhodeep ;
Ponda, Sameera S. ;
Wang, Ziyu .
NATURE, 2020, 588 (7836) :77-+
[7]   Marine Protected Area Networks in California, USA [J].
Botsford, Louis W. ;
White, J. Wilson ;
Carr, Mark H. ;
Caselle, Jennifer E. .
MARINE MANAGED AREAS AND FISHERIES, 2014, 69 :205-251
[8]   Target Search Control of AUV in Underwater Environment With Deep Reinforcement Learning [J].
Cao, Xiang ;
Sun, Changyin ;
Yan, Mingzhong .
IEEE ACCESS, 2019, 7 :96549-96559
[9]   Adaptive low-level control of autonomous underwater vehicles using deep reinforcement learning [J].
Carlucho, Ignacio ;
De Paula, Mariano ;
Wang, Sen ;
Petillot, Yvan ;
Acosta, Gerardo G. .
ROBOTICS AND AUTONOMOUS SYSTEMS, 2018, 107 :71-86
[10]   A behavior-based scheme using reinforcement learning for autonomous underwater vehicles [J].
Carreras, M ;
Yuh, J ;
Batlle, J ;
Ridao, P .
IEEE JOURNAL OF OCEANIC ENGINEERING, 2005, 30 (02) :416-427