Measuring the accuracy of gridded human population density surfaces: A case study in Bioko Island, Equatorial Guinea

Abstract

Background Geospatial datasets of population are becoming more common in models used for health policy. Publicly-available maps of human population make a consistent picture from inconsistent census data, and the techniques they use to impute data makes each population map unique. Each mapping model explains its methods, but it can be difficult to know which map is appropriate for which policy work. High quality census datasets, where available, are a unique opportunity to characterize maps by comparing them with truth. Methods We use census data from a bed-net mass-distribution campaign on Bioko Island, Equatorial Guinea, conducted by the Bioko Island Malaria Elimination Program as a gold standard to evaluate LandScan (LS), WorldPop Constrained (WP-C) and WorldPop Unconstrained (WP-U), Gridded Population of the World (GPW), and the High-Resolution Settlement Layer (HRSL). Each layer is compared to the gold-standard using statistical measures to evaluate distribution, error, and bias. We investigated how map choice affects burden estimates from a malaria prevalence model. Results Specific population layers were able to match the gold-standard distribution at different population densities. LandScan was able to most accurately capture highly urban distribution, HRSL and WP-C matched best at all other lower population densities. GPW and WP-U performed poorly everywhere. Correctly capturing empty pixels is key, and smaller pixel sizes (100 m vs 1 km) improve this. Normalizing areas based on known district populations increased performance. The use of differing population layers in a malaria model showed a disparity in results around transition points between endemicity levels. Discussion The metrics in this paper, some of them novel in this context, characterize how these population maps differ from the gold standard census and from each other. We show that the metrics help understand the performance of a population map within a malaria model. The closest match to the census data would combine LandScan within urban areas and the HRSL for rural areas. Researchers should prefer particular maps if health calculations have a strong dependency on knowing where people are not, or if it is important to categorize variation in density within a city.

From auxiliary data to research prospects, a review of gridded population mapping

“…Different choices of population grid datasets can significantly impact the applications (Leyk et al, 2019). Although some studies have attempted to compare the quality (mainly in terms of accuracy) of population grid products (Calka & Bielecka, 2019; Chen et al, 2020; Fries et al, 2021; Yin et al, 2021), the study areas and products tested in these studies are very limited. Therefore, it is difficult to answer which product is the best in a general sense.…”

“…Most developed countries in Europe and the United States already use community‐level “rolling online survey” data or a combination of administrative registration and census data for annual updates of fine‐scale demographic data (Zhu et al, 2020). There have been studies attempting to test the accuracy of the results using grid‐level demographic data (Calka & Bielecka, 2019; Fries et al, 2021). In this context, we are optimistic about fine‐scale demographic data available for future GPM studies.…”

“…There have been studies attempting to test the accuracy of the results using grid-level demographic data (Calka & Bielecka, 2019;Fries et al, 2021). In this context, we are optimistic about fine-scale demographic data available for future GPM studies.…”

Small area population denominators for improved disease surveillance and response

“…This is often a challenge to assess, as where detailed and recent population data exists, often it is the input to the models, leaving a lack of independent data to compare against. Studies using detailed census data ( Bustos et al, 2020 , Chen et al, 2020 , Fries et al, 2021 , Yin et al, 2021 ), as well as cross-validation ( Reed et al, 2018 , Stevens et al, 2015 , Stevens et al, 2020 ) have tried to assess how well different models replicate population numbers and distributions at the scale of available data, and unsurprisingly those more complex models using detailed settlement mapping and range of covariates tend to do best. However, multiple trade-offs exist in the production of such datasets that depend on aspects such as input data availability, geographical extent, temporal range, spatial resolution, intended use and user needs.…”

Small area population denominators for improved disease surveillance and response

“…This is often a challenge to assess, as where detailed, recent population data exist, often it is the input to the models, leaving a lack of independent data to compare against. Studies using detailed census data ( Bustos et al, 2020 , Chen et al, 2020 , Fries et al, 2021 , Yin et al, 2021 ), as well as cross-validation ( Reed et al, 2018 , Stevens et al, 2015 , Stevens et al, 2020 ) have tried to assess how well different models replicate population numbers and distributions at the scale of available data, and unsurprisingly those more complex models using detailed settlement mapping and range of covariates tend to do best. However, multiple trade-offs exist in the production of such datasets that depend on aspects such as input data availability, geographical extent, temporal range, spatial resolution, intended use and user needs.…”