The Verification of Ecological Citizen Science Data: Current approaches and future possibilities

Abstract

Citizen science schemes (projects) enable ecological data collection over very large spatial and temporal scales, producing datasets of high value for both pure and applied research. However, the accuracy of citizen science data is often questioned, owing to issues surrounding data quality and verification, the process by which records are checked after submission for correctness. Verification is a critical process for ensuring data quality and for increasing trust in such datasets, but verification approaches vary considerably among schemes. Here, we systematically review approaches to verification across ecological citizen science schemes, which feature in published research, aiming to identify the options available for verification, and to examine factors that influence the approaches used (Baker et al. 2021). We reviewed 259 schemes and were able to locate verification information for 142 of those. Expert verification was most widely used, especially among longer-running schemes. Community consensus was the second most common verification approach, used by schemes such as Snapshot Serengeti (Swanson et al. 2016) and MammalWeb (Hsing et al. 2018). It was more common among schemes with a larger number of participants and where photos or video had to be submitted with each record. Automated verification was not widely used among the schemes reviewed. Schemes that used automation, such as eBird (Kelling et al. 2011) and Project FeederWatch (Bonter and Cooper 2012) did so in conjunction with other methods such as expert verification. Expert verification has been the default approach for schemes in the past, but as the volume of data collected through citizen science schemes grows and the potential of automated approaches develops, many schemes might be able to implement approaches that verify data more efficiently. We present an idealised system for data verification, identifying schemes where this hierachical system could be applied and the requirements for implementation. We propose a hierarchical approach in which the bulk of records are verified by automation or community consensus, and any flagged records can then undergo additional levels of verification by experts.

Citizen Science and Monitoring Forest Pests: a Beneficial Alliance?

“…It can also be supported through the verification of records by experts, or it can be automated, e.g. using image recognition where photographs have been submitted [ 126 ]. Data verification provides an opportunity for feedback to volunteers.…”

Large‐scale mammal monitoring: The potential of a citizen science camera‐trapping project in the United Kingdom

“…A major challenge in citizen science data is reliability of observations including image classifications (Baker et al., 2021). With large numbers of classifications per image sequence, consensus approaches can yield high confidence regarding the species pictured (Swanson et al., 2016).…”

“…Even then, however, the relative lack of variety in the pictured fauna for the United Kingdom and the high incidence of images lacking mammals can deter many potential contributors. These points provide strong incentives to develop and refine automated approaches to image classification and verification (Baker et al., 2021; Green et al., 2020; see further, below).…”

An operational workflow for producing periodic estimates of species occupancy at large scales

“…In the UK we are fortunate in that biological recording has a wide taxonomic coverage: there are more than 80 schemes and societies, each focussing on the compilation and review of records for a taxonomic group of interest (Baker et al, 2021;Pocock et al, 2015). Through collaboration with these schemes, we have access to >24 million records for >10,000 species of bryophyte, lichen, insect and non-insect invertebrate (noting that many species are removed downstream; see Data manipulation).…”