Twice Is Nice: The Benefits of Two Ground Measures for Evaluating the Accuracy of Satellite-Based Sustainability Estimates

Abstract

Satellite data offer great promise for improving measures related to sustainable development goals. However, assessing satellite estimates is complicated by the fact that traditional ground-based measures of these same outcomes are often very noisy, leading to underestimation of satellite performance. Here, we quantify the amount of noise in traditional measures for three commonly studied outcomes in prior work—agricultural yields, household asset ownership, and household consumption expenditures—and present a theoretical basis for properly characterizing satellite performance in the presence of noisy ground data. We find that for both yield and consumption, repeated ground measures often disagree with each other, with less than half of the variability in one ground measure captured by the other. Estimates of the performance of satellite measures, in terms of squared correlation (r2), which account for this noise in ground data are accordingly higher, and occasionally even double, the apparent performance based on a naïve comparison of satellite and ground measures. Our results caution against evaluating satellite measures without accounting for noise in ground data and emphasize the benefit of estimating that noise by collecting at least two independent ground measures.

Retrospective Predictions of Rice and Other Crop Production in Madagascar Using Soil Moisture and an NDVI-Based Calendar from 2010–2017

“…Although a district-level analysis would be ideal, as it is the most well-resolved, we find that the district-level relationships between production and water availability are too noisy for interpretation (Figure S16), likely due to errors in reported production data, crop calendars, and crop maps but whose influence was reduced by averaging over larger climatezones. Errors in the reported production data, for example, may degrade the correspondence between production and soil moisture [55]. Regional analyses by climate-zones allows for important spatial heterogeneity to be resolved, though, that is not apparent in nationally reported statistics or predictions.…”

The integration of socio-economic and agricultural surveys by national statistical offices: The case of the Uganda Harmonized Integrated Survey1

“…In addition, the enumerator will establish at random, following the aforementioned MAPS protocols, two 8 m × 8 m sub-plots on each selected maize plot, for crop cutting purposes. The decision to set up two crop cut sub-plots is anchored in the evidence provided by Lobell et al [39] regarding the utility of at least two crop cut sub-plots for validating satellite-based maize estimates.…”

Optimal index insurance and basis risk decomposition: an application to Kenya

“…In a third iteration of the model, Jin et al (2019) report an accuracy of 0.5 against average yields at the county level. Although there is room for progress, these numbers are encouraging, as predicting yields in smallholder systems with high heterogeneity of practices and very small field size is particularly challenging, and errors in ground‐truth data will lead to underestimation of satellite performance (Lobell et al, 2021). Here we use estimates from a fourth iteration of the model, which were provided by the company Atlas AI.…”