Scaling whale monitoring using deep learning: A human-in-the-loop solution for analyzing aerial datasets

被引:7
作者
Boulent, Justine [1 ]
Charry, Bertrand [1 ]
Kennedy, Malcolm McHugh [1 ]
Tissier, Emily [1 ]
Fan, Raina [1 ]
Marcoux, Marianne [2 ]
Watt, Cortney A. [2 ]
Gagne-Turcotte, Antoine [1 ]
机构
[1] Whale Seeker, Montreal, PQ, Canada
[2] Fisheries & Oceans Canada, Aquat Res Div, Winnipeg, MB, Canada
关键词
semantic segmentation; automated cetacean detection; active learning; wildlife monitoring; artificial intelligence; MONODON-MONOCEROS; IDENTIFICATION; BIAS;
D O I
10.3389/fmars.2023.1099479
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
To ensure effective cetacean management and conservation policies, it is necessary to collect and rigorously analyze data about these populations. Remote sensing allows the acquisition of images over large observation areas, but due to the lack of reliable automatic analysis techniques, biologists usually analyze all images by hand. In this paper, we propose a human-in-the-loop approach to couple the power of deep learning-based automation with the expertise of biologists to develop a reliable artificial intelligence assisted annotation tool for cetacean monitoring. We tested this approach to analyze a dataset of 5334 aerial images acquired in 2017 by Fisheries and Oceans Canada to monitor belugas (Delphinapterus leucas) from the threatened Cumberland Sound population in Clearwater Fjord, Canada. First, we used a test subset of photographs to compare predictions obtained by the fine-tuned model to manual annotations made by three observers, expert marine mammal biologists. With only 100 annotated images for training, the model obtained between 90% and 91.4% mutual agreement with the three observers, exceeding the minimum inter-observer agreement of 88.6% obtained between the experts themselves. Second, this model was applied to the full dataset. The predictions were then verified by an observer and compared to annotations made completely manually and independently by another observer. The annotating observer and the human-in-the-loop pipeline detected 4051 belugas in common, out of a total of 4572 detections for the observer and 4298 for our pipeline. This experiment shows that the proposed human-in-the-loop approach is suitable for processing novel aerial datasets for beluga counting and can be used to scale cetacean monitoring. It also highlights that human observers, even experienced ones, have varied detection bias, underlining the need to discuss standardization of annotation protocols.
引用
收藏
页数:13
相关论文
共 38 条
[1]   Weakly Supervised Detection of Marine Animals in High Resolution Aerial Images [J].
Berg, Paul ;
Maia, Deise Santana ;
Pham, Minh-Tan ;
Lefevre, Sebastien .
REMOTE SENSING, 2022, 14 (02)
[2]   Aerial-trained deep learning networks for surveying cetaceans from satellite imagery [J].
Borowicz, Alex ;
Le, Hieu ;
Humphries, Grant ;
Nehls, Georg ;
Hoeschle, Caroline ;
Kosarev, Vladislav ;
Lynch, Heather J. .
PLOS ONE, 2019, 14 (10)
[3]   BIAS IN AERIAL SURVEY [J].
CAUGHLEY, G .
JOURNAL OF WILDLIFE MANAGEMENT, 1974, 38 (04) :921-933
[4]   Mapping Arctic cetaceans from space: A case study for beluga and narwhal [J].
Charry, Bertrand ;
Tissier, Emily ;
Iacozza, John ;
Marcoux, Marianne ;
Watt, Cortney A. .
PLOS ONE, 2021, 16 (08)
[5]   Hierarchical Classification of Narwhal Subpopulations Using Social Distance [J].
Charry, Bertrand ;
Marcoux, Marianne ;
Cardille, Jeffrey A. ;
Giroux-Bougard, Xavier ;
Humphries, Murray M. .
JOURNAL OF WILDLIFE MANAGEMENT, 2020, 84 (02) :311-319
[6]   Aerial photographic identification of narwhal (Monodon monoceros) newborns and their spatial proximity to the nearest adult female [J].
Charry, Bertrand ;
Marcoux, Marianne ;
Humphries, Murray M. .
ARCTIC SCIENCE, 2018, 4 (04) :513-524
[7]   Whales from space: Four mysticete species described using new VHR satellite imagery [J].
Cubaynes, Hannah C. ;
Fretwell, Peter T. ;
Bamford, Connor ;
Gerrish, Laura ;
Jackson, Jennifer A. .
MARINE MAMMAL SCIENCE, 2019, 35 (02) :466-491
[8]   Artificial intelligence and automated monitoring for assisting conservation of marine ecosystems: A perspective [J].
Ditria, Ellen M. ;
Buelow, Christina A. ;
Gonzalez-Rivero, Manuel ;
Connolly, Rod M. .
FRONTIERS IN MARINE SCIENCE, 2022, 9
[9]   Using Aerial Photogrammetry to Assess Stock-Wide Marine Turtle Nesting Distribution, Abundance and Cumulative Exposure to Industrial Activity [J].
Fossette, Sabrina ;
Loewenthal, Graham ;
Peel, Lauren R. ;
Vitenbergs, Anna ;
Hamel, Melanie A. ;
Douglas, Corrine ;
Tucker, Anton D. ;
Mayer, Florian ;
Whiting, Scott D. .
REMOTE SENSING, 2021, 13 (06)
[10]   Whales from Space: Counting Southern Right Whales by Satellite [J].
Fretwell, Peter T. ;
Staniland, Iain J. ;
Forcada, Jaume .
PLOS ONE, 2014, 9 (02)