Flukebook: an open-source AI platform for cetacean photo identification

Abstract

Determining which species are at greatest risk, where they are most vulnerable, and what are the trajectories of their communities and populations is critical for conservation and management. Globally distributed, wide-ranging whales and dolphins present a particular challenge in data collection because no single research team can record data over biologically meaningful areas. Flukebook.org is an open-source web platform that addresses these gaps by providing researchers with the latest computational tools. It integrates photo-identification algorithms with data management, sharing, and privacy infrastructure for whale and dolphin research, enabling the global collaborative study of these global species. With seven automatic identification algorithms trained for 15 different species, resulting in 37 species-specific identification pipelines, Flukebook is an extensible foundation that continually incorporates emerging AI techniques and applies them to cetacean photo identification through continued collaboration between computer vision researchers, software engineers, and biologists. With over 2.0 million photos of over 52,000 identified individual animals submitted by over 250 researchers, the platform enables a comprehensive understanding of cetacean populations, fostering international and cross-institutional collaboration while respecting data ownership and privacy. We outline the technology stack and architecture of Flukebook, its performance on real-world cetacean imagery, and its development as an example of scalable, extensible, and reusable open-source conservation software. Flukebook is a step change in our ability to conduct large-scale research on cetaceans across biologically meaningful geographic ranges, to rapidly iterate population assessments and abundance trajectories, and engage the public in actions to protect them.

Ocean nomads or island specialists? Culturally driven habitat partitioning contrasts in scale between geographically isolated sperm whale populations

“…A quality rating (Q) from 1 to 5 was given to each sperm whale fluke photograph [ 38 , 40 ] and only photographs with Q ≥ 3 were included in this analysis. All flukes (even those that were initially assigned in the field) were matched against known Caribbean individuals using the software Flukebook [ 41 ] ( ). Newly identified individuals were considered part of the same group (i.e.…”

Sperm and northern bottlenose whale interactions with deep-water trawlers in the western North Atlantic

“…Where the same photographic perspective on an identifying mark could be compared (e.g., tail fluke), individuals were identified across encounters. Sperm whale fluke photographs were added to Flukebook to confirm unique IDs within this dataset using computer assisted algorithms designed for sperm whales (Blount et al 2022). We calculated the minimum speed of the identified sperm whales by dividing the distance between two consecutive resights dividing by the elapsed time between them.…”

Artificial intelligence for right whale photo identification: from data science competition to worldwide collaboration

“…fluke book. org) is a flexible platform that integrates multiple computer vision algorithms for cetacean photo identification (Blount et al 2022). Cetaceans often have large spatial distributions spanning international boundaries, and therefore, cetacean conservation frequently requires collaboration between diverse stakeholders.…”

“…Neural networks are an established family of machine learning models able to learn and perform tasks by adjusting a network of artificial neurons that respond to an input layer, hidden layer(s), and then output to the next layer. Convolutional neural networks (CNN) are a variant of neural networks that have been successfully applied to computer vision problems, such as image classification (He et al 2016;Ma et al 2018;Tan et al 2019), image segmentation (Long et al 2015;Ronneberger et al 2015;Girshick et al 2018), object detection (Ren et al 2015;Redmon et al 2016;Hu et al 2017), microscopy (Xing et al 2017), and for photo identification of humans (Taigman et al 2014;Zhang et al 2017;Phillips et al 2018) and other animals (Crall et al 2013;Bogucki et al 2019;Moskvyak et al 2019;Weideman et al 2020;Cheeseman et al 2022;Clapham et al 2022;de Silva et al 2022;Blount et al 2022). As the number of layers in the neural networks increased, the term "deep learning" was coined (Dechter 1986;Aizenberg et al 2000).…”