Species‐level image classification with convolutional neural network enables insect identification from habitus images

Abstract

1. Changes in insect biomass, abundance, and diversity are challenging to track at sufficient spatial, temporal, and taxonomic resolution. Camera traps can capture habitus images of ground-dwelling insects. However, currently sampling involves manually detecting and identifying specimens. Here, we test whether a convolutional neural network (CNN) can classify habitus images of ground beetles to species level, and estimate how correct classification relates to body size, number of species inside genera, and species identity.2. We created an image database of 65,841 museum specimens comprising 361 carabid beetle species from the British Isles and fine-tuned the parameters of a pretrained CNN from a training dataset. By summing up class confidence values within genus, tribe, and subfamily and setting a confidence threshold, we trade-off between classification accuracy, precision, and recall and taxonomic resolution.3. The CNN classified 51.9% of 19,164 test images correctly to species level and 74.9% to genus level. Average classification recall on species level was 50.7%.Applying a threshold of 0.5 increased the average classification recall to 74.6% at the expense of taxonomic resolution. Higher top value from the output layer and larger sized species were more often classified correctly, as were images of species in genera with few species. 4. Fine-tuning enabled us to classify images with a high mean recall for the whole test dataset to species or higher taxonomic levels, however, with high variability.This indicates that some species are more difficult to identify because of properties such as their body size or the number of related species.5. Together, species-level image classification of arthropods from museum collections and ecological monitoring can substantially increase the amount of occurrence data that can feasibly be collected. These tools thus provide new opportunities in understanding and predicting ecological responses to environmental change. Jens-Christian Svenninghttps://orcid.

Deep learning and computer vision will transform entomology

“…Applications of gradient-weighted class activation mapping can even visualize morphologically important features for CNN classification (81). Classification accuracy is generally much lower when the insects are recorded live in their natural environments (82,83), but when class confidence is low at the species level, it may still be possible to confidently classify insects to a coarser taxonomic resolution (84). In recent years, impressive results have been obtained by CNNs (85).…”

“…It is common for ecological communities to contain a large fraction of relatively rare species. This often results in highly imbalanced datasets, and the number of specimens representing the rarest species could be insufficient for training neural networks (83,84). As such, advancing the development of algorithms and approaches for improved identification of rare classes is a key challenge for deep learning-based taxonomic identification (25).…”

Deep learning and computer vision will transform entomology

“…Applications of gradient-weighted class activation mapping can even visualize morphologically important features for CNN classification (84). Classification accuracy is generally much lower when the insects are recorded live in their natural environments (85,86), but when class confidence is low at the species-level, it may still be possible to confidently classify insects to a coarser taxonomic resolution (87). In recent years, impressive results have been obtained by CNNs (88).…”

“…In most ecological communities, it is common for species to be rare. This often results in highly imbalanced datasets, and the number of specimens representing the rarest species could be insufficient for training neural networks (86,87). As such, advancing the development of algorithms and approaches for improved identification of rare classes is a key challenge for deep learning-based taxonomic identification.…”

“…Species identification performance can vary widely, ranging from species which are correctly identified in most cases to species that are generally difficult to identify (96). Typically, the amount of training data is a key element for successful identification, although recent analyses of images of the approximately 65,000 specimens in the carabid beetle collection at the Natural History Museum London suggest that imbalances in identification performance are not necessarily related to how well-represented a species is in the training data (87). Further work is needed on large datasets to fully understand these challenges.…”