Accurate image-based identification of macroinvertebrate specimens using deep learning—How much training data is needed?

Abstract

Image-based methods for species identification offer cost-efficient solutions for biomonitoring. This is particularly relevant for invertebrate studies, where bulk samples often represent insurmountable workloads for sorting, identifying, and counting individual specimens. On the other hand, image-based classification using deep learning tools have strict requirements for the amount of training data, which is often a limiting factor. Here, we examine how classification accuracy increases with the amount of training data using the BIODISCOVER imaging system constructed for image-based classification and biomass estimation of invertebrate specimens. We use a balanced dataset of 60 specimens of each of 16 taxa of freshwater macroinvertebrates to systematically quantify how classification performance of a convolutional neural network (CNN) increases for individual taxa and the overall community as the number of specimens used for training is increased. We show a striking 99.2% classification accuracy when the CNN (EfficientNet-B6) is trained on 50 specimens of each taxon, and also how the lower classification accuracy of models trained on less data is particularly evident for morphologically similar species placed within the same taxonomic order. Even with as little as 15 specimens used for training, classification accuracy reached 97%. Our results add to a recent body of literature showing the huge potential of image-based methods and deep learning for specimen-based research, and furthermore offers a perspective to future automatized approaches for deriving ecological data from bulk arthropod samples.

Towards reliable estimates of abundance trends using automated non‐lethal moth traps

“…Automated monitoring systems must be as costefficient as possible to achieve the needed spatial coverage and resolution (Hahn et al, 2022). For AI implementation, training data needs to be collected and labelled in sufficient amounts, which can be expensive, especially under in situ conditions (Høye et al, 2022).…”

“…Automated monitoring systems must be as cost‐efficient as possible to achieve the needed spatial coverage and resolution (Hahn et al, 2022). For AI implementation, training data needs to be collected and labelled in sufficient amounts, which can be expensive, especially under in situ conditions (Høye et al, 2022). Moreover, the development from prototype to fully functional monitoring tool requires an adaptive process whose applicability and functionality in the real world is constantly re‐evaluated and adjusted accordingly (Hahn et al, 2022).…”

The Riverine Organism Drift Imager: A new technology to study organism drift in rivers and streams

“…Machine-learning-based identification methods for benthic invertebrates are well established (Ärje et al, 2020;Høye et al, 2022;Lytle et al, 2010;Raitoharju et al, 2018). However, they rely on preserved organisms to allow detailed image generation in laboratory conditions, and hence depend on labour-intensive, net-based sampling.…”

“…Recent advances in camera and machine‐learning technologies have enabled novel image‐based methods, which provide vast amounts of data and enable ecologists to study organisms cost‐effectively and non‐invasively (Høye et al, 2021; Lürig et al, 2021). Machine‐learning‐based identification methods for benthic invertebrates are well established (Ärje et al, 2020; Høye et al, 2022; Lytle et al, 2010; Raitoharju et al, 2018). However, they rely on preserved organisms to allow detailed image generation in laboratory conditions, and hence depend on labour‐intensive, net‐based sampling.…”

A gentle introduction to computer vision‐based specimen classification in ecological datasets

“…The exact sources of quasi-replication will vary greatly depending on the model's use case, but separating the training and test datasets by observer, location and/or time might be sensible (see section 3.2 of Van Horn et al, 2018 for a useful illustration of this concept).In general, the accuracy of a model will increase with the volume of data used to train the model. While the minimum required amount of training data will vary depending on the algorithm being used and the quality of the image data, a general 'rule of thumb' is ~50 specimens per class when there are few classes(Høye et al, 2022).However, as the total number of classes, data domains and species morphs increases, more training data are necessary. Additionally, it is ideal to have relatively equal representation of specimens per class in the dataset, as unevenness can introduce bias to the model (e.g Miao, 2021;Schneider et al, 2020)…”