Quantifying uncertainty in remotely sensed land cover maps

Cripps, Edward; O’Hagan, Anthony; Quaife, Tristan

doi:10.1007/s00477-012-0660-3

Cited by 5 publications

(9 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Quantification of uncertainty for accuracy metrics is not well implemented, and accuracy is often not reported together with any sort of dispersion measures (e.g., variance, confidence intervals) [16,17]. There is also critique about the non-spatial nature of the error matrix and that overall accuracy as a single measure does not represent how accuracy varies in space [11,37,38] or explain spatial patterns that are visible in the map. Debate is found around what should be the appropriate spatial assessment unit to evaluate accuracy (i.e., pixels, cluster/block of pixels, and/or polygons) [39] and the effects that different sampling designs (e.g., sample distribution of sampling units and sample size) might have on accuracy and precision [25,40,41].…”

Section: Introductionmentioning

confidence: 99%

Trends in Remote Sensing Accuracy Assessment Approaches in the Context of Natural Resources

et al. 2019

View full text Add to dashboard Cite

The utility of land cover maps for natural resources management relies on knowing the uncertainty associated with each map. The continuous advances typical of remote sensing, including the increasing availability of higher spatial and temporal resolution satellite data and data analysis capabilities, have created both opportunities and challenges for improving the application of accuracy assessment. There are well established accuracy assessment methods, but their underlying assumptions have not changed much in the last couple decades. Consequently, revisiting how map error and accuracy have been performed and reported over the last two decades is timely, to highlight areas where there is scope for better utilization of emerging opportunities. We conducted a quantitative literature review on accuracy assessment practices for mapping via remote sensing classification methods, in both terrestrial and marine environments. We performed a structured search for land and benthic cover mapping, limiting our search to journals within the remote sensing field, and papers published between 1998–2017. After an initial screening process, we assembled a database of 282 papers, and extracted and standardized information on various components of their reported accuracy assessments. We discovered that only 56% of the papers explicitly included an error matrix, and a very limited number (14%) reported overall accuracy with confidence intervals. The use of kappa continues to be standard practice, being reported in 50.4% of the literature published on or after 2012. Reference datasets used for validation were collected using a probability sampling design in 54% of the papers. For approximately 11% of the studies, the sampling design used could not be determined. No association was found between classification complexity (i.e. number of classes) and measured accuracy, independent from the size of the study area. Overall, only 32% of papers included an accuracy assessment that could be considered reproducible; that is, they included a probability-based sampling scheme to collect the reference dataset, a complete error matrix, and provided sufficient characterization of the reference datasets and sampling unit. Our findings indicate that considerable work remains to identify and adopt more statistically rigorous accuracy assessment practices to achieve transparent and comparable land and benthic cover maps.

show abstract

Section: Introductionmentioning

confidence: 99%

Trends in Remote Sensing Accuracy Assessment Approaches in the Context of Natural Resources

et al. 2019

View full text Add to dashboard Cite

show abstract

“…This represents a somewhat conservative estimate in the sense that it is likely to over-estimate the errors globally. Experiments with other values of d (not shown here) exhibited only small sensitivities to the choice of parameter value (See [24] for a discussion on the modelling of the spatial error probabilities and the robustness of the results for different values of d).…”

Section: Simulating the Distribution Of True Land Cover Classesmentioning

confidence: 91%

“…Here we only describe the components required to illustrate the implementation of the simulation scheme and the analysis of uncertainty of the GlobCover data. The reader is directed to [24] for more details on the statistical method and assumptions.…”

Section: Simulating the Distribution Of True Land Cover Classesmentioning

confidence: 99%

“…The output of the simulation scheme can then be used as inputs to any model that requires land cover data as an input, and can consequently estimate likely errors in model outputs due to uncertainties in land cover information. The statistical method is described in detail in [24], where it is demonstrated on a relatively small number of pixels representing a handful of PFTs. In this article, the method is applied to the GlobCover 2009 data set at the global scale, using the full range of land cover classes.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bayesian Analysis of Uncertainty in the GlobCover 2009 Land Cover Product at Climate Model Grid Scale

Quaife

Cripps

2016

Remote Sensing

Self Cite

View full text Add to dashboard Cite

Land cover data derived from satellites are commonly used to prescribe inputs to models of the land surface. Since such data inevitably contains errors, quantifying how uncertainties in the data affect a model's output is important. To do so, a spatial distribution of possible land cover values is required to propagate through the model's simulation. However, at large scales, such as those required for climate models, such spatial modelling can be difficult. Also, computer models often require land cover proportions at sites larger than the original map scale as inputs, and it is the uncertainty in these proportions that this article discusses. This paper describes a Monte Carlo sampling scheme that generates realisations of land cover proportions from the posterior distribution as implied by a Bayesian analysis that combines spatial information in the land cover map and its associated confusion matrix. The technique is computationally simple and has been applied previously to the Land Cover Map 2000 for the region of England and Wales. This article demonstrates the ability of the technique to scale up to large (global) satellite derived land cover maps and reports its application to the GlobCover 2009 data product. The results show that, in general, the GlobCover data possesses only small biases, with the largest belonging to non-vegetated surfaces. In vegetated surfaces, the most prominent area of uncertainty is Southern Africa, which represents a complex heterogeneous landscape. It is also clear from this study that greater resources need to be devoted to the construction of comprehensive confusion matrices.

show abstract

“…Unfortunately, sufficient sub-regional data are rarely available to support this (Strahler et al 2006). Cripps et al (2013) presented a Bayesian method for quantifying the uncertainty that results from potential misclassification in remotely sensed land cover maps. Discrete remote sensing classification neglects intrinsically & François Waldner patrick.bogaert@uclouvain.be the fuzzy character of the land surface and, as a consequence, leads to the inclusion of uncertainty in class assignments (Van der Wel et al 1998).…”

Section: Introductionmentioning

confidence: 99%

An information-based criterion to measure pixel-level thematic uncertainty in land cover classifications

Bogaert

Waldner

Defourny

2016

Stoch Environ Res Risk Assess

View full text Add to dashboard Cite

Traditional accuracy assessment of satellitederived maps relies on a confusion matrix and its associated indices built by comparing ground truth observations and classification outputs at specific locations. These indices may be applied at the map-level or at the class level. However, the spatial variation of the accuracy is not captured by those statistics. Pixel-level thematic uncertainty measures derived from class membership probability vectors can provide such spatially explicit information. In this paper, a new information-based criterion-the equivalent reference probability-is introduced to provide a synoptic thematic uncertainty measure that has the advantage of taking the maximum probability value into account while committing for the full set of probabilities. The fundamental theoretical properties of this indicator was first highlighted and its use was afterwards demonstrated on a real case study in Belgium. Results showed that the proposed approach positively correlates with the quality of the classification and is more sensitive than the classical maximum probability criterion. As this information-based criterion can be used for providing spatially explicit maps of thematic uncertainty quality, it provides substantial additional information regarding classification quality compared to conventional quality measures. Accordingly, it proved to be useful both for end-users and map producers as a way to better understand the nature of the errors and to subsequently improve the map quality.

show abstract

Quantifying uncertainty in remotely sensed land cover maps

Cited by 5 publications

References 37 publications

Trends in Remote Sensing Accuracy Assessment Approaches in the Context of Natural Resources

Trends in Remote Sensing Accuracy Assessment Approaches in the Context of Natural Resources

Bayesian Analysis of Uncertainty in the GlobCover 2009 Land Cover Product at Climate Model Grid Scale

An information-based criterion to measure pixel-level thematic uncertainty in land cover classifications

Contact Info

Product

Resources

About