On the use of the overlapping area matrix for image segmentation evaluation: A survey and new performance measures

Ortiz, Alberto; Oliver, Gabriel

doi:10.1016/j.patrec.2006.05.002

Cited by 35 publications

(18 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This method finds oneto-one matches between the reference and output objects. To be able to handle over-detections where more than one output object correspond to a reference object, and under-detections where more than one reference object correspond to an output object, the maximum overlap criterion can be relaxed to allow all overlaps above a certain threshold (Hoover et al, 1996;Mariano et al, 2002;Ortiz and Oliver, 2006). Alternatively, Jiang et al (2006) used maximum-weight bipartite graph matching to find optimal one-to-one matching between the reference and output objects where the weights correspond to overlaps among the objects.…”

Section: Related Work On Object Detection Evaluationmentioning

confidence: 99%

Performance measures for object detection evaluation

Ozdemir

Aksoy

Eckert

et al. 2010

Pattern Recognition Letters

View full text Add to dashboard Cite

a b s t r a c tWe propose a new procedure for quantitative evaluation of object detection algorithms. The procedure consists of a matching stage for finding correspondences between reference and output objects, an accuracy score that is sensitive to object shapes as well as boundary and fragmentation errors, and a ranking step for final ordering of the algorithms using multiple performance indicators. The procedure is illustrated on a building detection task where the resulting rankings are consistent with the visual inspection of the detection maps.

show abstract

Section: Related Work On Object Detection Evaluationmentioning

confidence: 99%

Performance measures for object detection evaluation

Ozdemir

Aksoy

Eckert

et al. 2010

Pattern Recognition Letters

View full text Add to dashboard Cite

show abstract

“…In Baraldi, Boschetti, and Humber (2014), a crisp thematic map assessment protocol was proposed based on: (i) a probability sampling strategy, (ii) a pair of test and reference thematic map legends A and B that may differ, (iii) a crisp overlapping area matrix, OAMTRX = FrequencyCount(A × B), defined as a BIVRFTAB instantiation estimated from a geospatial population with or without sampling, (Beauchemin & Thomson, 1997; Ortiz & Oliver, 2006), whose spatial unit x is (0D) pixel, (iv) a set of thematic quantitative quality indicators (Q 2 Is), TQ 2 Is, extracted from the OAMTRX and (v) a set of spatial Q 2 Is (SQ 2 Is) extracted from sub-symbolic image-objects (image-segments) in the multi-level map domain, where image-objects are either (0D) pixels, (1D) lines or (2D) polygons according to the OGC nomenclature (OGC, 2015). Whereas the construction of an OAMTRX is straightforward and non-controversial when categorical labels of sampling units are crisp (hard), the method to construct an OAMTRX when categorical labels are soft (fuzzy) is not obvious at all; for example, refer to (Kuzera & Pontius, 2008).…”

Section: Methodsmentioning

confidence: 99%

GEO-CEOS stage 4 validation of the Satellite Image Automatic Mapper lightweight computer program for ESA Earth observation level 2 product generation – Part 2: Validation

et al. 2018

View full text Add to dashboard Cite

ESA defines as Earth Observation (EO) Level 2 information product a multi-spectral (MS) image corrected for atmospheric, adjacency, and topographic effects, stacked with its data-derived scene classification map (SCM), whose legend includes quality layers cloud and cloud-shadow. No ESA EO Level 2 product has ever been systematically generated at the ground segment. To fill the information gap from EO big data to ESA EO Level 2 product in compliance with the GEO-CEOS stage 4 validation (Val) guidelines, an off-the-shelf Satellite Image Automatic Mapper (SIAM) lightweight computer program was selected to be validated by independent means on an annual 30 m resolution Web-Enabled Landsat Data (WELD) image composite time-series of the conterminous U.S. (CONUS) for the years 2006 to 2009. The SIAM core is a prior knowledge-based decision tree for MS reflectance space hyperpolyhedralization into static (non-adaptive to data) color names. For the sake of readability, this paper was split into two. The present Part 2—Validation—accomplishes a GEO-CEOS stage 4 Val of the test SIAM-WELD annual map time-series in comparison with a reference 30 m resolution 16-class USGS National Land Cover Data (NLCD) 2006 map. These test and reference map pairs feature the same spatial resolution and spatial extent, but their legends differ and must be harmonized, in agreement with the previous Part 1 - Theory. Conclusions are that SIAM systematically delivers an ESA EO Level 2 SCM product instantiation whose legend complies with the standard 2-level 4-class FAO Land Cover Classification System (LCCS) Dichotomous Phase (DP) taxonomy.

show abstract

“…In greater detail, for any BIVRFTAB instance, either square or non-square, there is a binary relationship R: A ⇒ B ⊆ A × B that guides the interpretation process, where “correct” binary entry-pair cells of the 2-fold Cartesian product A × B are equal to 1 and located either off-diagonal (scattered) or on-diagonal, if a main diagonal exists when the BIVRFTAB is square. When a BIVRFTAB is estimated from a geospatial population with or without sampling, it is called overlapping area matrix (OAMTRX) (Baraldi et al, 2014; Baraldi, Bruzzone, & Blonda, 2005; Baraldi et al, 2006; Beauchemin & Thomson, 1997; Lunetta & Elvidge, 1999; Ortiz & Oliver, 2006; Pontius & Connors, 2006). When the binary relationship R: A ⇒ B is a bijective function (both 1–1 and onto), i.e., when the two categorical variables A and B estimated from a single population coincide, then the BIVRFTAB instantiation is square and sorted; it is typically called confusion matrix (CMTRX) or error matrix (Congalton & Green, 1999; Lunetta & Elvidge, 1999; Pontius & Millones, 2011; Stehman & Czaplewski, 1998).…”

Section: Problem Background Of Color Naming In Cognitive Sciencementioning

confidence: 99%

GEO-CEOS stage 4 validation of the Satellite Image Automatic Mapper lightweight computer program for ESA Earth observation level 2 product generation - Part 1: Theory

et al. 2018

View full text Add to dashboard Cite

ESA defines as Earth Observation (EO) Level 2 information product a single-date multi-spectral (MS) image corrected for atmospheric, adjacency and topographic effects, stacked with its data-derived scene classification map (SCM), whose legend includes quality layers cloud and cloud-shadow. No ESA EO Level 2 product has ever been systematically generated at the ground segment. To fill the information gap from EO big data to ESA EO Level 2 product in compliance with the GEO-CEOS stage 4 validation (Val) guidelines, an off-the-shelf Satellite Image Automatic Mapper (SIAM) lightweight computer program was validated by independent means on an annual 30 m resolution Web-Enabled Landsat Data (WELD) image composite time-series of the conterminous U.S. (CONUS) for the years 2006–2009. The SIAM core is a prior knowledge-based decision tree for MS reflectance space hyperpolyhedralization into static color names. Typically, a vocabulary of MS color names in a MS data (hyper)cube and a dictionary of land cover (LC) class names in the scene-domain do not coincide and must be harmonized (reconciled). The present Part 1—Theory provides the multidisciplinary background of a priori color naming. The subsequent Part 2—Validation accomplishes a GEO-CEOS stage 4 Val of the test SIAM-WELD annual map time-series in comparison with a reference 30 m resolution 16-class USGS National Land Cover Data 2006 map, based on an original protocol for wall-to-wall thematic map quality assessment without sampling, where the test and reference maps feature the same spatial resolution and spatial extent, but whose legends differ and must be harmonized.

show abstract

On the use of the overlapping area matrix for image segmentation evaluation: A survey and new performance measures

Cited by 35 publications

References 39 publications

Performance measures for object detection evaluation

Performance measures for object detection evaluation

GEO-CEOS stage 4 validation of the Satellite Image Automatic Mapper lightweight computer program for ESA Earth observation level 2 product generation – Part 2: Validation

GEO-CEOS stage 4 validation of the Satellite Image Automatic Mapper lightweight computer program for ESA Earth observation level 2 product generation - Part 1: Theory

Contact Info

Product

Resources

About