Designing online species identification tools for biological recording: the impact on data quality and citizen science learning

Abstract

In recent years, the number and scale of environmental citizen science programmes that involve lay people in scientific research have increased rapidly. Many of these initiatives are concerned with the recording and identification of species, processes which are increasingly mediated through digital interfaces. Here, we address the growing need to understand the particular role of digital identification tools, both in generating scientific data and in supporting learning by lay people engaged in citizen science activities pertaining to biological recording communities. Starting from two well-known identification tools, namely identification keys and field guides, this study focuses on the decision-making and quality of learning processes underlying species identification tasks, by comparing three digital interfaces designed to identify bumblebee species. The three interfaces varied with respect to whether species were directly compared or filtered by matching on visual features; and whether the order of filters was directed by the interface or a user-driven open choice. A concurrent mixed-methods approach was adopted to compare how these different interfaces affected the ability of participants to make correct and quick species identifications, and to better understand how participants learned through using these interfaces. We found that the accuracy of identification and quality of learning were dependent upon the interface type, the difficulty of the specimen on the image being identified and the interaction between interface type and ‘image difficulty’. Specifically, interfaces based on filtering outperformed those based on direct visual comparison across all metrics, and an open choice of filters led to higher accuracy than the interface that directed the filtering. Our results have direct implications for the design of online identification technologies for biological recording, irrespective of whether the goal is to collect higher quality citizen science data, or to support user learning and engagement in these communities of practice.

Evaluating the ability of citizen scientists to identify bumblebee (Bombus) species

“…We expected identification ability to improve over time, as a result of verification feedback and learner experience [8,30,32,43]. We found that the accuracy of bumblebee identifications varied throughout the year in both projects, but did not show an improvement across months, probably because different bumblebee species emerge and are active at different times.…”

“…Recorders are encouraged to use the BeeWatch website resources, including a simple binomial key, to identify their bumblebee. All records are verified/corrected by experts at the University of Aberdeen or BBCT and automated feedback is provided to the recorder [30–32].…”

“…In Blooms for Bees, this consisted of any comments that the verifier regarded as helpful, usually within two to three weeks of the submission. In BeeWatch this involved automated natural language generated feedback within four weeks of submission [30–32].…”

Error rates in wildlife image classification

“…The experiment by Thornton et al strongly suggests that training and providing classifiers with detailed visual aids improves image classification, echoing results from Kosmala et al () and Sharma, Colucci‐Gray, Siddharthan, Comont, and Wal (). We still do predict that inexperienced and untrained undergraduate students would have poorer agreement compared to the experts used in our study if both groups examined the same images.…”

Maximising the potential for citizen science in New South Wales

“…Citizen scientist #2 processed 7,483 images, had 86.7% (4703 / 5422) accuracy identifying animal species, and 90.4% (6764 / 7483) accuracy in identifying the presence of an animal and then the correct species. Variability in human accuracy is an area that is starting to be examined more fully (Sharma et al 2019) in an attempt to improve accuracy for citizen science projects. For example, is the difference in accuracy a reflection of time taken when reviewing images or expertise?…”