No one-size-fits-all solution to clean GBIF

Abstract

28Species occurrence records provide the basis for many biodiversity studies. They derive from geo-referenced specimens deposited in natural history collections and visual observations, such as those obtained through various mobile applications. Given the rapid increase in availability of such data, the control of quality and accuracy constitutes a particular concern. Automatic flagging and filtering are a scalable and reproducible means to identify potentially problematic records in datasets from public databases such as the Global Biodiversity Information Facility (GBIF; www.gbif.org). However, it is unclear how much data may be lost by filtering, whether the same tests should be applied across all taxonomic groups, and what is the effect of filtering for common downstream analyses. Here, we evaluate the effect of 13 recently proposed filters on the inference of species richness patterns and automated conservation assessments for 18 Neotropical taxa including animals, fungi, and plants, terrestrial and marine, downloaded from GBIF. We find that 29-90% of the records are potentially erroneous, with large variation across taxonomic groups. Tests for duplicated information, collection year, basis of record as well as urban areas and coordinates for terrestrial taxa in the sea or marine taxa on land have the greatest effect. While many flagged records might not be de facto erroneous, they could be overly imprecise and increase uncertainty in downstream analyses. Automated flagging can help in identifying problematic records, but requires customization of which tests and thresholds should be applied to the taxonomic group and geographic area under focus. Our results stress the importance of thorough exploration of the meta-data associated with species records for biodiversity research. 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44Publicly available species distribution data have become a crucial resource in biodiversity research, including studies in 46 ecology, biogeography, systematics and conservation biology. In particular, the availability of digitized collections from 47 museums and herbaria, and citizen science observations has increased drastically over the last few years. As of today, 48 the largest public aggregator for geo-referenced species occurrences data, the Global Biodiversity Information Facility 49 (www.gbif.org), provides access to more than 1.3 billion geo-referenced occurrence records for species from across the 50 globe and the tree of life. 51A central challenge to the use of these publicly available species occurrence data in research are erroneous geographic 52 coordinates (Anderson et al. 2016). Errors mostly arise because public databases integrate records collected with 53 different methodologies in different places, at different times; often without centralized curation and only rudimentary 54 meta-data. For instance, erroneous coordinates caused by data-entry errors or automated geo-referencing from vague 55 locality descriptions are common (Maldonado et al. 2015; Yesson et al. 2007)...

Automated conservation assessment of the orchid family with deep learning

“…Data quality. We compared the accuracy of ConR at three levels of data curation of the occurrence records (1) "raw", the data downloaded from GBIF scrubbed taxonomically, (2) "intermediate", the raw data subjected to automated removal of records with common geographic errors (Zizka et al 2020b), and (3) "filtered", the intermediate data with additional removal of records outside the known occurrence range of species from WCSP (WCSP 2019). We only ran this test with ConR, since we expect the two other index-based methods to yield similar results and because we expect IUC-NN to be more robust against erroneous coordinates, since the features used for IUC-NN prediction are mean values across all records of a species.…”

Automated conservation assessment of the orchid family using deep learning

“…We compared the accuracy of ConR assessments based on different datasets of species occurrences, representing three levels of data curation (1) "raw", the data downloaded from GBIF scrubbed taxonomically, (2) "intermediate", the raw data subjected to automated removal of records with common geographic errors (Zizka et al 2020b), and (3) "filtered", the intermediate data with additional removal of records outside the known occurrence range of species from WCSP (WCSP 2019). We only ran this test with ConR, since we expect the two other index-based methods to respond similarly to the issue and because we expect IUC-NN to be more robust against erroneous coordinates, since the features used for IUC-NN prediction are mean values across all records of a species.…”

“…3) How robust are AA to limitations with data availability, variable data quality and geographic sampling biases? Digitally available species occurrence records are often biased towards certain geographic regions (Meyer et al 2016;Zizka et al 2020a) and contain erroneous or very imprecise coordinates (Zizka et al 2019(Zizka et al , 2020b. Therefore, we expect a bias of AA towards well-sampled geographic areas and life forms, and an increase in accuracy with geographic cleaning.…”

Increasing accuracy: The advantage of using open access species occurrence database in the Red List assessment

“…Compared to Praptosuwiryo (2020) as the case study in our paper, the number of occurrence records stored in GBIF is more than three times of that in Praptosuwiryo (2020) who mainly used data from Herbarium Bogoriense. The use of GBIF records in Red List assessment, however, has to be performed carefully as the data might have errors in the geographic coordinates (Zizka et al 2020). Therefore, cleaning the data is necessary before mapping the occurrence points and conducting the assessment.…”

“…To use these occurrence records, data cleaning is an essential first step as the data might contain errors in the geographic coordinates. While manual data cleaning based on expert knowledge is feasible on small taxonomic or geographic areas, automated flagging methods can be used for a huge dataset of occurrence records (Zizka et al 2020).…”