A review of accuracy assessment for object-based image analysis: From per-pixel to per-polygon approaches

Su, Ye; Pontius, Robert Gilmore; Rakshit, Rahul

doi:10.1016/j.isprsjprs.2018.04.002

Cited by 148 publications

(141 citation statements)

References 110 publications

Supporting

Mentioning

134

Contrasting

Unclassified

Order By: Relevance

“…To ensure the accuracy of assessment, 640 reference points representing 40% of the training sample were used to represent all land use classes. Then, Google Earth images were used as reference data to access the accuracy of classified maps [39,40].…”

Section: Land Use/cover Changesmentioning

confidence: 99%

Quantifying Surface Urban Heat Island Formation in the World Heritage Tropical Mountain City of Sri Lanka

Ranagalage

Dissanayake

Murayama

et al. 2018

IJGI

View full text Add to dashboard Cite

Presently, the urban heat island (UHI) phenomenon, and its adverse impacts, are becoming major research foci in various interrelated fields due to rapid changes in urban ecological environments. Various cities have been investigated in previous studies, and most of the findings have facilitated the introduction of proper mitigation measures to overcome the negative impact of UHI. At present, most of the mountain cities of the world have undergone rapid urban development, and this has resulted in the increasing surface UHI (SUHI) phenomenon. Hence, this study focuses on quantifying SUHI in Kandy City, the world heritage tropical mountain city of Sri Lanka, using Landsat data (1996 and 2017) based on the mean land surface temperature (LST), the difference between the fraction of impervious surfaces (IS), and the fraction of green space (GS). Additionally, we examined the relationship of LST to the green space/impervious surface fraction ratio (GS/IS fraction ratio) and the magnitude of the GS/IS fraction ratio. The SUHI intensity (SUHII) was calculated based on the temperature difference between main land use/cover categories and the temperature difference between urban-rural zones. We demarcated the rural zone based on the fraction of IS recorded, <10%, along with the urban-rural gradient zone. The result shows a SUHII increase from 3.9 • C in 1996 to 6.2 • C in 2017 along the urban-rural gradient between the urban and rural zones (10 < IS). These results relate to the rapid urban expansion of the study areas from 1996 to 2017. Most of the natural surfaces have changed to impervious surfaces, causing an increase of SUHI in Kandy City. The mean LST has a positive relationship with the fraction of IS and a negative relationship with the fraction of GS. Additionally, the GS/IS fraction ratio shows a rapid decline. Thus, the findings of this study can be considered as a proxy indicator for introducing proper landscape and urban planning for the World Heritage tropical mountain city of Kandy in Sri Lanka.

show abstract

Section: Land Use/cover Changesmentioning

confidence: 99%

Quantifying Surface Urban Heat Island Formation in the World Heritage Tropical Mountain City of Sri Lanka

Ranagalage

Dissanayake

Murayama

et al. 2018

IJGI

View full text Add to dashboard Cite

show abstract

“…Similar to the previous works in the literature [36,37], we randomly select 80% images from each class to train the SVM model and the remained 20% images are used for testing. According to [43], overall accuracy and confusion matrix are usually adopted as the metrics for accuracy assessment. Other related works such as [44] also report measures that derive from the confusion matrix, in which the Bradley-Terry Model was used to quantify association in remotely sensed images.…”

Section: Results For Remote Sensing Scene Classificationmentioning

confidence: 99%

Region-Wise Deep Feature Representation for Remote Sensing Images

et al. 2018

View full text Add to dashboard Cite

Effective feature representations play an important role in remote sensing image analysis tasks. With the rapid progress of deep learning techniques, deep features have been widely applied to remote sensing image understanding in recent years and shown powerful ability in image representation. The existing deep feature extraction approaches are usually carried out on the whole image directly. However, such deep feature representation strategies may not effectively capture the local geometric invariance of target regions in remote sensing images. In this paper, we propose a novel region-wise deep feature extraction framework for remote sensing images. First, regions that may contain the target information are extracted from one whole image. Then, these regions are fed into a pre-trained convolutional neural network (CNN) model to extract regional deep features. Finally, the regional deep features are encoded by an improved Vector of Locally Aggregated Descriptors (VLAD) algorithm to generate the feature representation for the image. We conducted extensive experiments on remote sensing image classification and retrieval tasks based on the proposed region-wise deep feature extraction framework. The comparison results show that the proposed approach is superior to the existing CNN feature extraction methods.

show abstract

“…The area under this curve (AUC) of an ROC/TOC plot is often used as a single measure of overall accuracy that summarizes numerous thresholds for the continuous variable [96]. There are also metrics for assessing the accuracy of object-based image analysis (OBIA, [97]), which we do not cover here (but see the supplementary information (SI)) because the choice of measure varies according to mapping objectives [65,98].…”

Section: Map Accuracy Assessment Proceduresmentioning

confidence: 99%

“…However, in practice this accounting is rarely done, while map reference data uncertainty is also rarely examined [34,38,57]. This tendency is illustrated by Ye et al [65], who reviewed 209 journal articles focused on object-based image analysis, finding that one third gave incomplete information about the sample design and size of their map reference data, let alone any mention of error within the sample. Errors in map reference data can bias the map accuracy assessment [47,99], as well as estimates derived from the confusion matrix, such as land cover class proportions and their standard errors [46].…”

Section: Map Accuracy Assessment Proceduresmentioning

confidence: 99%

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

et al. 2020

Self Cite

View full text Add to dashboard Cite

Remote sensing, or Earth Observation (EO), is increasingly used to understand Earth system dynamics and create continuous and categorical maps of biophysical properties and land cover, especially based on recent advances in machine learning (ML). ML models typically require large, spatially explicit training datasets to make accurate predictions. Training data (TD) are typically generated by digitizing polygons on high spatial-resolution imagery, by collecting in situ data, or by using pre-existing datasets. TD are often assumed to accurately represent the truth, but in practice almost always have error, stemming from (1) sample design, and (2) sample collection errors. The latter is particularly relevant for image-interpreted TD, an increasingly commonly used method due to its practicality and the increasing training sample size requirements of modern ML algorithms. TD errors can cause substantial errors in the maps created using ML algorithms, which may impact map use and interpretation. Despite these potential errors and their real-world consequences for map-based decisions, TD error is often not accounted for or reported in EO research. Here we review the current practices for collecting and handling TD. We identify the sources of TD error, and illustrate their impacts using several case studies representing different EO applications (infrastructure mapping, global surface flux estimates, and agricultural monitoring), and provide guidelines for minimizing and accounting for TD errors. To harmonize terminology, we distinguish TD from three other classes of data that should be used to create and assess ML models: training reference data, used to assess the quality of TD during data generation; validation data, used to iteratively improve models; and map reference data, used only for final accuracy assessment. We focus primarily on TD, but our advice is generally applicable to all four classes, and we ground our review in established best practices for map accuracy assessment literature. EO researchers should start by determining the tolerable levels of map error and appropriate error metrics. Next, TD error should be minimized during sample design by choosing a representative spatio-temporal collection strategy, by using spatially and temporally relevant imagery and ancillary data sources during TD creation, and by selecting a set of legend definitions supported by the data. Furthermore, TD error can be minimized during the collection of individual samples by using consensus-based collection strategies, by directly comparing interpreted training observations against expert-generated training reference data to derive TD error metrics, and by providing image interpreters with thorough application-specific training. We strongly advise that TD error is incorporated in model outputs, either directly in bias and variance estimates or, at a minimum, by documenting the sources and implications of error. TD should be fully documented and made available via an open TD repository, allowing others to replicate and assess its use. To guide researchers in this process, we propose three tiers of TD error accounting standards. Finally, we advise researchers to clearly communicate the magnitude and impacts of TD error on map outputs, with specific consideration given to the likely map audience.

show abstract

A review of accuracy assessment for object-based image analysis: From per-pixel to per-polygon approaches

Cited by 148 publications

References 110 publications

Quantifying Surface Urban Heat Island Formation in the World Heritage Tropical Mountain City of Sri Lanka

Quantifying Surface Urban Heat Island Formation in the World Heritage Tropical Mountain City of Sri Lanka

Region-Wise Deep Feature Representation for Remote Sensing Images

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

Contact Info

Product

Resources

About