Good practices for estimating area and assessing accuracy of land change

Abstract

1The remote sensing science and applications communities have developed increasingly reliable, 2 consistent, and robust approaches for capturing land dynamics to meet a range of information 3 needs. Statistically robust and transparent approaches for assessing accuracy and estimating area 4 of change are critical to ensure the integrity of land change information. We provide 5 practitioners with a set of "good practice" recommendations for designing and implementing an 6 accuracy assessment of a change map and estimating area based on the reference sample data. 7The good practice recommendations address the three major components: of the process 8 including the sampling design, response design and analysis. The primary good practice 9 recommendations for assessing accuracy and estimating area are: (i) implement a probability 10 sampling design that is chosen to achieve the priority objectives of accuracy and area estimation 11 while also satisfying practical constraints such as cost and available sources of reference data; 12(ii) implement a response design protocol that is based on reference data sources that provide 13 sufficient spatial and temporal representation to accurately label each unit in the sample (i.e., the 14 "reference classification" will be considerably more accurate than the map classification being 15 evaluated); (iii) implement an analysis that is consistent with the sampling design and response 16 design protocols; (iv) summarize the accuracy assessment by reporting the estimated error matrix 17 in terms of proportion of area and estimates of overall accuracy, user's accuracy (or commission 18 error), and producer's accuracy (or omission error); (v) estimate area of classes (e.g., types of 19 change such as wetland loss or types of no changepersistence such as stable forest) based on the 20 reference classification of the sample units; (vi) quantify uncertainty by reporting confidence 21 intervals for accuracy and area parameters; (vii) evaluate variability and potential error in the 22 3 reference classification; and (viii) document deviations from good practice that may substantially 23 affect the results. An example application is provided to illustrate the recommended process. 24 4

Change of Land Use/Cover in Tianjin City Based on the Markov and Cellular Automata Models

“…To access the reliability of the resulting LULC maps, an accuracy assessment was performed [35,36]. The accuracy of the land-use classification was checked with 500 random sample points for Built-up, Cropland, Grass, Forest and Water [37].…”

Detection of olive oil mill waste (OOMW) disposal areas using high resolution GeoEye’s OrbView-3 and Google Earth images

“…This is in part because the promoted standard methods such as the Kappa coefficient are not always appropriate [45]. Several researchers have worked on the problems relative to accuracy assessment of classification uncertainty [46][47][48]. This paper provides established accuracy measures for OOMW areas.…”

Comparing Quantity, Allocation and Configuration Accuracy of Multiple Land Change Models

“…The lack of studies comparing models with identical input data limits our ability to understand the trade-offs among different model's accuracy and therefore constitutes a critical research gap. There is considerable research to quantitatively guide researchers in understanding how accurate a model simulation is in terms of its quantity and allocation [12][13][14][15][16][17], but few that evaluate model accuracy in terms of landscape configuration. Configuration refers to the spatial arrangement of landscape types and their shapes.…”

“…Historically, the most common method to assess simulation results was visual inspection by experts [18,24,25], however this is highly subjective and irreproducible. Several methods have been proposed over the last decade, all of which compare simulated maps with maps of assumed truth [12][13][14][15][16][17]26]. There is continued debate over accuracy assessment methodologies among land change and remote sensing research foci, with the sole emphasis on quantity and allocation disagreement [13,14].…”