Evaluating the ability of citizen scientists to identify bumblebee (Bombus) species

Abstract

Citizen science is an increasingly popular way of engaging volunteers in the collection of scientific data. Despite this, data quality remains a concern and there is little published evidence about the accuracy of records generated by citizen scientists. Here we compare data generated by two British citizen science projects, Blooms for Bees and BeeWatch, to determine the ability of volunteer recorders to identify bumblebee ( Bombus ) species. We assessed recorders’ identification ability in two ways–as recorder accuracy (the proportion of expert-verified records correctly identified by recorders) and recorder success (the proportion of recorder-submitted identifications confirmed correct by verifiers). Recorder identification ability was low (<50% accuracy; <60% success), despite access to project specific bumblebee identification materials. Identification ability varied significantly depending on bumblebee species, with recorders most able to correctly identify species with distinct appearances. Blooms for Bees recorders (largely recruited from the gardening community) were markedly less able to identify bumblebees than BeeWatch recorders (largely individuals with a more specific interest in bumblebees). Within both projects, recorders demonstrated an improvement in identification ability over time. Here we demonstrate and quantify the essential role of expert verification within citizen science projects, and highlight where resources could be strengthened to improve recorder ability.

Participatory science and directed survey methods: A case study with odontocetes in the Maui Nui region of the Hawaiian Islands

“…Sharing unusual sightings with all volunteers or creating reporting incentives, such as prizes for the most sightings (Hochachka et al, 2012), may help maintain reporting levels for future projects. Species identification is of critical importance to this study and identification ability may not be consistent across all volunteers (Falk et al, 2019); mistakes may be more prevalent for species of similar appearance (e.g. spinner and spotted dolphins).…”

Community science participants gain environmental awareness and contribute high quality data but improvements are needed: insights from Bumble Bee Watch

“…This suggests that although more can be done, such as customized regional bee identification workshops or pollinator-friendly gardening guides, program organizers are on the right track with the provided features. We do not discuss the accuracy or perception of user and expert identification in this paper as has been done for other bumble bee community science programs (Lye et al, 2011;Suzuki-Ohno et al, 2017;Falk et al, 2019), but this information (including commonly mis-identified species) would also help to improve the program and data quality (but see MacPhail et al (unpublished data) for this).…”

“…It was launched in 2014 with a web-based platform (http://www.bumblebeewatch.org), with mobile applications for iOS and Android launched in 2017 and 2018 respectively. Based on the idea that bumble bees may, in many cases, be identifiable to species from photographs (Lye et al, 2011;Suzuki-Ohno et al, 2017;Falk et al, 2019), participants take photos of bumble bees and submit through the web or app, along with a date observed and location information. They have the option to use an interactive key or smart filter to assign species name (pre-limited by location), and regional experts verify the identifications.…”

Using Bumble Bee Watch to investigate the accuracy and perception of bumble bee (Bombus spp.) identification by community scientists

“…Community science, which is also known as citizen science, is a popular tool in conservation biology that involves public participation in scientific data collection (Silvertown, 2009;Crall et al, 2011;Kremen, Ullmann & Thorp, 2011;Lye et al, 2011;Dickinson et al, 2012;Lebuhn et al, 2012;Roy et al, 2012;Birkin & Goulson, 2015;Kobori et al, 2016;Kosmala et al, 2016;McKinley et al, 2017). The main strengths of community science include increasing the spatial and temporal scale and magnitude of sampling efforts; reducing the cost of sampling; and creating educational and recreational benefits for participants (Bonney et al, 2009;Kremen, Ullmann & Thorp, 2011;Dickinson et al, 2012;Lebuhn et al, 2012;Birkin & Goulson, 2015;McKinley et al, 2017;Falk et al, 2019).…”

“…Previous research has found that community scientist data, when unreviewed or unverified by experts, can contain errors that significantly influence the interpretation of results by experts (Gardiner et al, 2012;Comont & Ashbrook, 2017). Problematic errors include the overestimation of rare or at-risk species, as well as underestimation of common species, inflated species richness and significant increases in species diversity (Dickinson, Zuckerberg & Bonter, 2010;Gardiner et al, 2012;Silvertown et al, 2013;Comont & Ashbrook, 2017;Falk et al, 2019).…”