Species identification by conservation practitioners using online images: accuracy and agreement between experts

Abstract

Emerging technologies have led to an increase in species observations being recorded via digital images. Such visual records are easily shared, and are often uploaded to online communities when help is required to identify or validate species. Although this is common practice, little is known about the accuracy of species identification from such images. Using online images of newts that are native and non-native to the UK, this study asked holders of great crested newt (Triturus cristatus) licences (issued by UK authorities to permit surveying for this species) to sort these images into groups, and to assign species names to those groups. All of these experts identified the native species, but agreement among these participants was low, with some being cautious in committing to definitive identifications. Individuals’ accuracy was also independent of both their experience and self-assessed ability. Furthermore, mean accuracy was not uniform across species (69–96%). These findings demonstrate the difficulty of accurate identification of newts from a single image, and that expert judgements are variable, even within the same knowledgeable community. We suggest that identification decisions should be made on multiple images and verified by more than one expert, which could improve the reliability of species data.

Error rates in wildlife image classification

“…We note simply that all of the experts we sampled had conducted research or management sampling for at least one of the species, and they were in professional positions such that members of the public (or even others in their departments) would come to them with images for classification. We agree with Kosmala, Wiggins, Swanson, and Simmons () and Austen et al () that people vary in what they recognize as expertise, but we believe that the positions and experience held by the people we sampled would meet most such definitions.…”

“…The experiment by Thornton et al builds upon our work on species classification from camera‐trapping images, but their response minimizes our original findings and fails to recognize our take‐home point: image classification to the species level is often difficult for similar‐looking, sympatric species, and studies with such images should take extra measures to account for this challenge. The inconsistencies in image classification that we uncovered indicate that misclassification rates for similar‐looking species may be high and should be explicitly addressed during study design—points echoed by recent research on images of mice and rats (Burns, Parrott, Rowe, & Phillips, ), newts (Austen, Bindemann, Griffiths, & Roberts, ), and even for distinguishing between cougars ( Felis concolor ), bobcats, and house cats ( Felis catus , LaRue, ).…”

Quantifying data quality in a citizen science monitoring program: False negatives, false positives and occupancy trends

“…The first, which we can consider as sampling false-positives, arise when surveyors report seeing a species when it is not in fact present at (e.g., through misidentification or wrong field notes). However, we may consider a second source of false-positives, termed ecological false-positives by Berigan, Jones, Whitmore, Gutiérrez, and Peery (2019), when detections are also made of individuals which are temporarily making use of a site outside of their home range-for example, when individuals are dispersing (Sutherland, Elston, & Lambin, 2013), or foraging outside of their normal home range (Berigan et al, 2019). The presence of such observations will mean that traditional analysis provides inference on site-use, rather than occupancy patterns in the strict sense of the term (Altwegg & Nichols, 2019).…”

Use-inspired basic research on individual differences in face identification: implications for criminal investigation and security

“…In addition, people seem to have limited insight into their own face perception abilities (see Bindemann, Attard, & Johnston, 2014 ; Bobak, Mileva, & Hancock, in press ; Palermo et al, 2017 ; but see Gray, Bird, & Cook, 2017 ; Livingston & Shah, in press ; Shah, Gaule, Sowden, Bird, & Cook, 2015 ). This implies that experts’ self-reports also cannot be taken as evidence that they possess enhanced ability to identify faces (see also Austen, Bindemann, Griffiths, & Roberts, 2018 ).…”