SEALNET: Facial recognition software for ecological studies of harbor seals

Abstract

Methods for long‐term monitoring of coastal species such as harbor seals ( Phoca vitulina ) are often costly, time‐consuming, and highly invasive, underscoring the need for improved techniques for data collection and analysis. Here, we propose the use of automated facial recognition technology for identification of individual seals and demonstrate its utility in ecological and population studies. We created a software package, SealNet, that automates photo identification of seals, using a graphical user interface (GUI) software to detect, align, and chip seal faces from photographs and a deep convolutional neural network (CNN) suitable for small datasets (e.g., 100 seals with five photos per seal) to classify individual seals. We piloted the SealNet technology with a population of harbor seals located within Casco Bay on the coast of Maine, USA. Across two years of sampling, 2019 and 2020, at seven haul‐out sites in Middle Bay, we obtained a dataset optimized for the development and testing of SealNet. We processed 1752 images representing 408 individual seals and achieved 88% Rank‐1 and 96% Rank‐5 accuracy in closed set seal identification. In identifying individual seals, SealNet software outperformed a similar face recognition method, PrimNet, developed for primates but retrained on seals. The ease and wealth of image data that can be processed using SealNet software contributes a vital tool for ecological and behavioral studies of marine mammals in the developing field of conservation technology.

Individual identification of endangered amphibians using deep learning and smartphone images: case study of the Japanese giant salamander (Andrias japonicus)

“…The selection of the measurement region for identi cation is important and has a signi cant effect on the accuracy of the model. For example, in the case of seals, the accuracy was 59% for fur-based identi cation [37] but improved to 88% for face-based identi cation [17] . Arzoumanian et al [38] achieved more than 90% pair image matching using ank (front dorsal region) spot patterns for the identi cation of whale sharks but reported that image matching failed when photographs were obtained at oblique angles of more than 30°.…”

“…This technique, which has been used for human facial recognition, was rst applied for animal identi cation in 2014 [16] . The target species for individual identi cation are mainly mammals [17][18][19][20] ,…”

Growth and opportunities for drone surveillance in pinniped research

“…With visible‐light aerial imagery, deep learning techniques have already been applied to estimate aggregate pinniped counts (Hoekendijk et al 2021), detect individual pinnipeds (Dujon et al 2021), and classify pinnipeds by age class (Salberg 2015, Infantes et al 2022), though success and generalisability vary widely between examples. Upcoming applications also include deep learning for photogrammetry, as has already been demonstrated with drone‐based photography of cetaceans (Gray et al 2019) and more recently with harbour seals (Infantes et al 2022), and deep learning for individual identification, as has been demonstrated with ground‐based photography of harbour seals (Nepovinnykh et al 2018, 2022, Birenbaum et al 2022). In this early stage of its technological deployment, deep learning for computer vision remains an experimental technique in pinniped research, and still few examples characterise its error and generalisability across large‐scale applications.…”

“…Some current drone applications with pinnipeds leverage thermal or multispectral imagery to facilitate detection by high contrast in drone imagery (Seymour et al 2017, Sweeney et al 2019, Larsen et al 2022b, but many more studies rely exclusively on visible-light photography to detect pinnipeds. With visible-light aerial imagery, deep learning techniques have already been applied to estimate aggregate pinniped counts (Hoekendijk et al 2021) (Nepovinnykh et al 2018, Birenbaum et al 2022. In this early stage of its technological deployment, deep learning for computer vision remains an experimental technique in pinniped research, and still few examples characterise its error and generalisability across large-scale applications.…”

Individual identification of endangered amphibians using deep learning and smartphone images: case study of the Japanese giant salamander (Andrias japonicus)

“…This technique, which has been used for human facial recognition, was first applied to animal identification in 2014 16 . The target species for individual identification have mainly been mammals 17 – 20 , although the method has also been applied to birds and reptiles, with large research bias observed according to the taxonomic group 8 , 21 . Pattern recognition has been used to identify amphibian individuals 22 , but image recognition based on deep learning has not yet been applied.…”