Using Bumble Bee Watch to investigate the accuracy and perception of bumble bee (Bombus spp.) identification by community scientists

Abstract

Community science programs provide an opportunity to gather scientific data to inform conservation policy and management. This study examines the accuracy of community science identifications submitted to the North American Bumble Bee Watch program on a per species level and as compared to each species’ conservation status, as well as users (members of the public) and experts (those with expertise in the field of bumble bee biology) perceived ease of species identification. Photos of bumble bees (Hymenoptera: Apidae: Bombus) are submitted to the program by users and verified (species name corrected or assigned as necessary) by an expert. Over 22,000 records from over 4,900 users were used in the analyses. Accuracy was measured in two ways: percent agreement (percent of all records submitted correctly by users) and veracity (percent of all verified records submitted correctly by the users). Users generally perceived it harder to identify species than experts. User perceptions were not significantly different from the observed percent agreement or veracity, while expert perceptions were significantly different (overly optimistic) from the observed percent agreement but not the veracity. We compared user submitted names to final expert verified names and found that, for all species combined, the average percent agreement was 53.20% while the average veracity was 55.86%. There was a wide range in percent agreement values per species, although sample size and the role of chance did affect some species agreements. As the conservation status of species increased to higher levels of extinction risk, species were increasingly more likely to have a lower percent agreement but higher levels of veracity than species of least concern. For each species name submitted, the number of different species verified by experts varied from 1 to 32. Future research may investigate which factors relate to success in user identification through community science. These findings could play a role in informing the design of community science programs in the future, including for use in long-term and national-level monitoring of wild pollinators.

Assessing the potential for deep learning and computer vision to identify bumble bee species from images

“…With programs such as BeeSpotter (beespotter.org), Bumble Bee Watch (BBW; bumblebeewatch.org), iNaturalist (inaturalist.org), and the Wisconsin Bumble Bee Brigade (WBBB; wiatri.net/inventory/bbb), users can contribute to national and regional databases by uploading georeferenced images of bumble bees and providing preliminary identifications, which are then verified by experts or community sourced. This verification step is important because user-submitted identifications to WBBB and BBW agree with experts only 73 and 53 percent of the time, respectively 15 , 16 . If not properly verified, such erroneous data could have serious negative consequences for pollinator conservation.…”

Validating Morphometrics with DNA Barcoding to Reliably Separate Three Cryptic Species of Bombus Cresson (Hymenoptera: Apidae)

“…Goulson et al [ 49 ] point out that community science surveys are often limited by the taxonomic skills of the observers, particularly for bee species that are difficult or impossible to identify in the field because of cryptic coloration. MacPhail et al [ 50 ] reported that 46.3% of B. vagans , 38.6% of B. sandersoni , and 86.4% of B. perplexus were correctly identified from photos of Bombus by project designated expert taxonomists, or they were placed into a “two-striped species group” when the photos were ambiguous. Richardson et al [ 13 ] found that they could reliably make a species determination from 68% of photographs submitted by participants in the Vermont Bumble Bee Atlas.…”

Nectar concentrating behavior by bees (Hymenoptera: Anthophila)

“…Despite these oft-cited limitations, community science is an effective tool for gathering data over a large geographic scale (Deguines et al 2016) or detecting relatively rare events such as bee nesting (Lye et al 2012). Further, community science can provide more reliable identifications when observations are validated by multiple expert taxonomists (Le Féon et al 2016;Falk et al 2019;MacPhail et al 2020b). In this study, we addressed this issue by using embedded taxonomic experts to confirm bee identifications and ensuring appropriately conservative levels of taxonomic resolution (often subgenus-level) with species identifications made conservatively where warranted.…”