Rise of the Machines: Best Practices and Experimental Evaluation of Computer-Assisted Dorsal Fin Image Matching Systems for Bottlenose Dolphins

Moore, Reny B. Tyson; Urian, Kim W.; Allen, Jason B.; Cush, Carolyn; Parham, Jason; Blount, Drew; Holmberg, Jason; Thompson, Jamie W.; Wells, Randall S.

doi:10.3389/fmars.2022.849813

Cited by 6 publications

(2 citation statements)

References 57 publications

(126 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bergler et al, 2021; Maglietta et al, 2020, 2023; Thompson et al, 2021; Weideman et al, 2017). A full comparison (sensu Tyson Moore et al, 2022), however, would be challenging because we are unaware of any model that could be reasonably used to predict the identities of every species in the dataset. Thus, any comparison would need to be piecemeal, comparing the state‐of‐the‐art for each species to the full model presented here.…”

Section: Discussionmentioning

confidence: 99%

A deep learning approach to photo–identification demonstrates high performance on two dozen cetacean species

et al. 2023

Self Cite

View full text Add to dashboard Cite

Researchers can investigate many aspects of animal ecology through noninvasive photo–identification. Photo–identification is becoming more efficient as matching individuals between photos is increasingly automated. However, the convolutional neural network models that have facilitated this change need many training images to generalize well. As a result, they have often been developed for individual species that meet this threshold. These single‐species methods might underperform, as they ignore potential similarities in identifying characteristics and the photo–identification process among species. In this paper, we introduce a multi‐species photo–identification model based on a state‐of‐the‐art method in human facial recognition, the ArcFace classification head. Our model uses two such heads to jointly classify species and identities, allowing species to share information and parameters within the network. As a demonstration, we trained this model with 50,796 images from 39 catalogues of 24 cetacean species, evaluating its predictive performance on 21,192 test images from the same catalogues. We further evaluated its predictive performance with two external catalogues entirely composed of identities that the model did not see during training. The model achieved a mean average precision (MAP) of 0.869 on the test set. Of these, 10 catalogues representing seven species achieved a MAP score over 0.95. For some species, there was notable variation in performance among catalogues, largely explained by variation in photo quality. Finally, the model appeared to generalize well, with the two external catalogues scoring similarly to their species' counterparts in the larger test set. From our cetacean application, we provide a list of recommendations for potential users of this model, focusing on those with cetacean photo–identification catalogues. For example, users with high quality images of animals identified by dorsal nicks and notches should expect near optimal performance. Users can expect decreasing performance for catalogues with higher proportions of indistinct individuals or poor quality photos. Finally, we note that this model is currently freely available as code in a GitHub repository and as a graphical user interface, with additional functionality for collaborative data management, via Happywhale.com.

show abstract

Section: Discussionmentioning

confidence: 99%

A deep learning approach to photo–identification demonstrates high performance on two dozen cetacean species

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…The ability of a semiautomated photo‐ID program to identify individuals from their naturally occurring pigmentation has been tested in several taxa, including amphibians (Matthé et al 2017, Renet et al 2019), birds (Sherley et al 2010), fish (Speed et al 2007, González‐Ramos et al 2017), reptiles (Carter et al 2014, Dunbar et al 2014), terrestrial mammals (Bolger et al 2012, Nipko et al 2020), and marine mammals (Langley et al 2021, Tyson Moore et al 2022).…”

mentioning

confidence: 99%

Evaluating the performance of semiautomated photographic identification programs for leopard seals

Grabham,

van der Linde,

Nelson

2024

Wildlife Society Bulletin

View full text Add to dashboard Cite

The photographic identification (photo‐ID) of individual animals can be time‐consuming and erroneous. Recent efforts to collect photographs of leopard seals (Hydrurga leptonyx) from across their range have necessitated the technological streamlining of photo‐ID. We constructed a dataset containing 595 photographs of the head and body of individuals recorded in New Zealand between 2008 and 2022 to test the performance of semiautomated 3 photo‐ID programs: HotSpotter, Interactive Individual Identification System's Pattern+ (I3S), and Wild‐ID. We classified attributes of photographs (e.g., quality) and individuals (i.e., pelage patterns) to assess their effect on performance. We compared performance using Top20 and Top1 Accuracy, defined as the proportion of test photographs where the highest ranked correct identity was in the top 20 and top 1, respectively, matched reference photographs. HotSpotter outperformed I3S and Wild‐ID in both Top20 and Top1 Accuracy of most assessed attributes. Maximizing HotSpotter's performance may be achieved through several methods, including increasing the number and variety of photographs of individuals in the dataset. HotSpotter will likely perform better with photographs without obstructions (e.g., debris from beaches), such as on the pack ice of their primary Antarctic range. We highlight the viability of HotSpotter in assisting the photo‐ID of leopard seals, and more broadly, other species with similar markings.

show abstract

Species-Agnostic Patterned Animal Re-identification by Aggregating Deep Local Features

Nepovinnykh,

Chelak,

Eerola

et al. 2024

Int J Comput Vis

View full text Add to dashboard Cite

Access to large image volumes through camera traps and crowdsourcing provides novel possibilities for animal monitoring and conservation. It calls for automatic methods for analysis, in particular, when re-identifying individual animals from the images. Most existing re-identification methods rely on either hand-crafted local features or end-to-end learning of fur pattern similarity. The former does not need labeled training data, while the latter, although very data-hungry typically outperforms the former when enough training data is available. We propose a novel re-identification pipeline that combines the strengths of both approaches by utilizing modern learnable local features and feature aggregation. This creates representative pattern feature embeddings that provide high re-identification accuracy while allowing us to apply the method to small datasets by using pre-trained feature descriptors. We report a comprehensive comparison of different modern local features and demonstrate the advantages of the proposed pipeline on two very different species.

show abstract

Rise of the Machines: Best Practices and Experimental Evaluation of Computer-Assisted Dorsal Fin Image Matching Systems for Bottlenose Dolphins

Cited by 6 publications

References 57 publications

A deep learning approach to photo–identification demonstrates high performance on two dozen cetacean species

A deep learning approach to photo–identification demonstrates high performance on two dozen cetacean species

Evaluating the performance of semiautomated photographic identification programs for leopard seals

Species-Agnostic Patterned Animal Re-identification by Aggregating Deep Local Features

Contact Info

Product

Resources

About