Data reliability in citizen science: learning curve and the effects of training method, volunteer background and experience on identification accuracy of insects visiting ivy flowers

Abstract

210X.12581Abstract  Citizen science, the involvement of volunteers in collecting of scientific data, can be a useful research tool. However, data collected by volunteers are often of lower quality than that collected by professional scientists. We studied the accuracy with which volunteers identified insects visiting ivy (Hedera) flowers in Sussex, England. In the first experiment, we examined the effects of training method, volunteer background and prior experience. Fifty-three participants were trained for the same duration using one of three different methods (pamphlet, pamphlet + slide show, pamphlet + direct training). Almost immediately following training, we tested the ability of participants to identify live insects on ivy flowers to one of 10 taxonomic categories and recorded whether their identifications were correct or incorrect, without providing feedback. The results showed that the type of training method had a significant effect on identification accuracy (P = 0.008). Participants identified 79.1% of insects correctly after using a one-page colour pamphlet, 85.6% correctly after using the pamphlet and viewing a slide show, and 94.3% correctly after using the pamphlet in combination with direct training in the field. As direct training cannot be delivered remotely, in the following year we conducted a second experiment, in which a different sample of 26 volunteers received the pamphlet plus slide show training repeatedly three times. Moreover, in this experiment participants received c. 2 minutes of additional training material, either videos of insects or stills taken from the videos. Testing showed that identification accuracy increased from 88.6% to 91.3% to 97.5% across the three successive tests.We also found a borderline significant interaction between the type of additional 2 material and the test number (P = 0.053), such that the video gave fewer errors than stills in the first two tests only. The most common errors made by volunteers were misidentifications of honey bees and social wasps with their hover fly mimics. We also tested six experts who achieved nearly perfect accuracy (99.8%), which shows what is possible in practice. Overall, our study shows that two or three sessions of remote training can be as good as one of direct training, even for relatively challenging taxonomic discriminations that include distinguishing models and mimics. IntroductionCitizen science, in which volunteers collect scientific data, is an increasingly popular tool with great potential in research (Dickinson & Bonney 2012; Theobald et al. 2015). By However, citizen science has several challenges. In order for the data to be useful, volunteers generally need training, supervision and overall management. In addition, the data collected by volunteers is often perceived as of low quality and unreliable. For example, in 1993 an amendment was made to prohibit the US National Biological Survey from accepting the work of volunteers, following the assertion by the House of Representatives that "volunteers are i...

Identifying Barriers to Citizen Scientist Retention When Measuring Pollination Services

“…However, with training, citizen scientists can collect useful and accurate pollinator information Deguines et al 2012). Methods such as expert verification of volunteer-collected arthropod specimens (Gardiner et al 2012), hands-on training Ratnieks et al 2016), and/or assigning target species to groups based on color or pattern (Roy et al 2016) all reduce, although do not always eliminate, volunteer errors.…”

Linee Guida per il monitoraggio dei coleotteri saproxilici protetti in Europa

“…Questo dato è in accordo con Gardiner et al (2012) che rilevano come la CS applicata al monitoraggio delle coccinelle fornisce dati corretti nell'81-100% dei casi; Ratnieks et al (2016) hanno accertato che i volontari identificano correttamente fino al 94% di specie di insetti. Questi due autori hanno anche dimostrato come i metodi e i materiali impiegati per l'addestramento e il trasferimento di informazioni ai volontari abbiano avuto un ruolo determinante nell'accuratezza delle identificazioni.…”

Identifying Reliable Opportunistic Data for Species Distribution Modeling: A Benchmark Data Optimization Approach

“…Although professionally collected data are the preferred data source for SDM, they are expensive to collect and are often in short supply. Data collected using proper crowdsourcing techniques, often termed "opportunistic data" [3][4][5][6][7][8][9][10][11][12] or unstructured volunteer data, can provide ecologists with a variety of biodiversity monitoring data. Consequently, volunteer-based citizen science monitoring systems have attracted a lot of attention.…”

“…Munson et al [27] also found that the eBird opportunistic data had more uncertainty than the professionally collected North American Breeding Bird Survey (BBS) data. Due to these and other issues, many still consider opportunistic data to be low quality and unreliable for research and conservation planning purposes [9].…”

“…Collection-oriented techniques rely on data collection standards to validate opportunistic data, i.e., who uploaded the data and how were the data collected. One popular user-oriented technique that has spurred numerous related ecological sampling methodologies, considers the amount of filtered data extracted per species per specific site or visited location [9,17,24,33]. Other examples of collection-oriented techniques are those that incorporate meta-data on the relative expertise of data collectors, e.g., Yu et al [34].…”