Artificial intelligence for right whale photo identification: from data science competition to worldwide collaboration

Abstract

Photo identification is an important tool in the conservation management of endangered species, and recent developments in artificial intelligence are revolutionizing existing workflows to identify individual animals. In 2015, the National Oceanic and Atmospheric Administration hosted a Kaggle data science competition to automate the identification of endangered North Atlantic right whales (Eubalaena glacialis). The winning algorithms developed by Deepsense.ai were able to identify individuals with 87% accuracy using a series of convolutional neural networks to identify the region of interest, create standardized photographs of uniform size and orientation, and then identify the correct individual. Since that time, we have brought in many more collaborators as we moved from prototype to production. Leveraging the existing infrastructure by Wild Me, the developers of Flukebook, we have created a web-based platform that allows biologists with no machine learning expertise to utilize semi-automated photo identification of right whales. New models were generated on an updated dataset using the winning Deepsense.ai algorithms. Given the morphological similarity between the North Atlantic right whale and closely related southern right whale (Eubalaena australis), we expanded the system to incorporate the largest long-term photo identification catalogs around the world including the United States, Canada, Australia, South Africa, Argentina, Brazil, and New Zealand. The system is now fully operational with multi-feature matching for both North Atlantic right whales and southern right whales from aerial photos of their heads (Deepsense), lateral photos of their heads (Pose Invariant Embeddings), flukes (CurvRank v2), and peduncle scarring (HotSpotter). We hope to encourage researchers to embrace both broad data collaborations and artificial intelligence to increase our understanding of wild populations and aid conservation efforts.

Application of multi-species photo-ID database management systems: a user’s perspective

“…Each centralised catalogue can retain its own identification numbering system and be kept separately, while images from various catalogues can be compared with one another; a useful feature when working with species that exhibit long-range movements (e.g., Acevedo et al 2022;Genov et al 2022). However, as DISCOVERY does not offer automated image recognition, the crossmatching process is not as time-efficient as in recently developed AI-based matching systems (e.g., Cheeseman et al 2022;Khan et al 2022), especially when working with large datasets.…”

Relation between beluga whale aggregations and sea temperature on climate change forecasts

“…In remote sensing methods, free Google Earth images have been used to train Deep Convolutional Neural Networks (CNN) for conservation goals (Guirado et al, 2017), which automatically learn the distinctive features of each object class from a large set of annotated images (LeCun et al, 2015). Previous studies in marine mammal detection and counting have resulted in a recall rate of more than 80% (Bogucki et al, 2019;Guirado et al, 2019;Gabaldon et al, 2022;Khan et al, 2022). This can have a big impact in reducing the effort required for manual verification, increasing the advantage of employing an automatic detector in long-term monitoring (Li et al, 2022;Marquez et al, 2022), estimating population abundances and projecting dynamics and fluctuations under future climate change scenarios.…”

Individual identification and photographic techniques in mammalian ecological and behavioural research—Part 1: Methods and concepts

“…Under such circumstances, efficient data processing and management becomes a challenge. Lately, however, rapid advances in automated pattern-recognition and machine-learning algorithms have been catching up with current state-of-the-art digital photography, opening up this field of research to new possibilities and ideas of how to further improve the efficiency of field data collection (e.g., Schofield et al 2019;Butcher et al 2021;Machado and Cantor 2022), broaden the type, array and geographic scale of data collected during any given field day (e.g., Rieucau et al 2018;Schneider et al 2019;Khan et al 2022), and-importantly-improve the efficiency of processing the individual-ID data immediately upon the completion of a field day (e.g., Lahiri et al 2011;Guo et al 2020;) and ensure the reliability of identification systems across long time-scales (e.g., Bodesheim et al 2022;Cheeseman et al 2022;Tyson Moore et al 2022). Other photographic applications also received a fresh breath of new ideas, such as photogrammetry (e.g., Galimberti et al 2019;Gray et al 2019;Shirane et al 2020;O'Connell-Rodwell et al 2022;Richardson et al 2022) or biodiversity and monitoring surveys (e.g., Miao et al 2019;Tabak et al 2019;Howell et al 2022;Sun et al 2022;Rydell et al 2022), broadening their applications well beyond their customary use.…”

“…These developments have instigated novel means of data management that link different types of data (e.g., behavioral, genetic, geographic, and environmental) with individual-IDs and can facilitate a multi-purpose centralized database for several species and multiple study areas, with data collected at large geographical scales by multiple research teams (e.g., Gailey and Karczmarski 2012;Keen et al 2022). Some of the currently available systems offer various dynamic functions that can be tailored to suit project-specific requirements and user-specific needs, facilitating data processing that is faster and less prone to human error (e.g., Lahiri et al 2011;Miele et al 2021;Cheeseman et al 2022) and data sharing that promotes fruitful research collaboration (e.g., Berger-Wolf et al 2017;Blount et al 2022;Khan et al 2022).…”