Scaling whale monitoring using deep learning: A human-in-the-loop solution for analyzing aerial datasets

Abstract

To ensure effective cetacean management and conservation policies, it is necessary to collect and rigorously analyze data about these populations. Remote sensing allows the acquisition of images over large observation areas, but due to the lack of reliable automatic analysis techniques, biologists usually analyze all images by hand. In this paper, we propose a human-in-the-loop approach to couple the power of deep learning-based automation with the expertise of biologists to develop a reliable artificial intelligence assisted annotation tool for cetacean monitoring. We tested this approach to analyze a dataset of 5334 aerial images acquired in 2017 by Fisheries and Oceans Canada to monitor belugas (Delphinapterus leucas) from the threatened Cumberland Sound population in Clearwater Fjord, Canada. First, we used a test subset of photographs to compare predictions obtained by the fine-tuned model to manual annotations made by three Observers, expert marine mammal biologists. With only 100 annotated images for training, the model obtained between 90% and 91.4% mutual agreement with the three Observers, exceeding the minimum inter-observer agreement of 88.6% obtained between the experts themselves. Second, this model was applied to the full dataset. The predictions were then verified by an Observer and compared to annotations made completely manually and independently by another Observer. The annotating Observer and the human-in-the-loop pipeline detected 4051 belugas in common, out of a total of 4572 detections for the Observer and 4298 for our pipeline. This experiment shows that the proposed human-in-the-loop approach is suitable for processing novel aerial datasets for beluga counting and can be used to scale cetacean monitoring. It also highlights that human observers, even experienced ones, have varied detection bias, underlining the need to discuss standardization of annotation protocols.

Relevance of Data Analytics in Sustainable Fisheries Management: An Evidence-Based Study

“…In the high sea benthic zones, machine learning methods e.g. Random Forest (RF), and Support Vector Regression (SVR) are used to estimate water turbidity, and chlorophyll (micro and macro algae blooms) (Ashphaq et al, 2023), and to precisely and effectively monitor marine mammals-cetaceans, especially in Canada (Boulent et al, 2023). This is done mainly by using a robust analysis of massive datasets of photographs of cetaceans combined deep learning models and dimensionality reduction methods, for instance, via the creation of a binary land cover map, thus creating a more effective and precise method of monitoring cetaceans thanks to the application of data analysis/analytics techniques.…”

Seas the DNA? Limited detection of cetaceans by low‐volume environmental DNA transect surveys

“…More common methods for monitoring cetacean presence mostly consist of a combination of visual and acoustic techniques (Cartagena‐Matos et al., 2021; Dalpaz et al., 2021; Liu et al., 2022), including aerial and satellite surveillance (Boulent et al., 2023; Charry et al., 2021). Each of these approaches includes inherent biases to data collection; acoustic monitoring can be spatially limited and focused toward only a portion of species depending on the device used (Barkley et al., 2021; Liu et al., 2022; Rice et al., 2021), whereas visual boat‐based surveys that cover a wider spatial area than acoustic monitoring, however, are limited to when animals surface to breathe, time of day and year, visibility, weather, sea state, observer bias, and vessel avoidance bias (Dalpaz et al., 2021; Forney et al., 1991; Marsh & Sinclair, 1989; Oliveira‐Rodrigues et al., 2022).…”