<title>Metrics for image segmentation</title>

Abstract

An important challenge in mapping image-processing techniques onto applications is the lack of quantitative performance measures. From a systems engineering perspective these are essential if system level requirements are to be decomposed into sub-system requirements which can be understood in terms of algorithm selection and performance optimisation.Nowhere in computer vision is this more evident than in the area of image segmentation. This is a vigorous and innovative research activity, but even after nearly two decades of progress, it remains almost impossible to answer the question "what would the performance of this segmentation algorithm be under these new conditions?" -To begin to address this shortcoming, we have devised a well-principled metric for assessing the relative performance of two segmentation algorithms. This allows meaningful objective comparisons to be made between their outputs. It also estimates the absolute performance of an algorithm given ground truth. Our approach is an information theoretic one. In this paper, we describe the theory and motivation of our method, and present practical results obtained from a range of state of the art segmentation methods. We demonstrate that it is possible to measure the objective performance of these algorithms, and to use the information so gained to provide clues about how their performance might be improved.

A computationally efficient evaluation environment for image segmentation

“…The smaller Sowerby database [9] is an earlier collection of hand-segmented images which has been utilized to evaluate edge-detection algorithms [10]. The work in [11] also employed ground truth from a set of fifty images and applied analysis of variance to five evaluation metrics for object recognition algorithms.…”

Computationally Efficient Quantitative Testing of Image Segmentation with a Genetic Algorithm

“…The smaller Sowerby database [15] is an earlier collection of hand-segmented images which has been used to evaluate edge-detection algorithms [16]. The work in [17] also used ground truth from a set of fifty images and applied analysis of variance to five evaluation metrics for object recognition algorithms.…”

<title>Using quantitative statistics for the construction of machine vision systems</title>

“…For instance, problems can arise when the output data of one algorithm is to be fed into several subsequent algorithms, each having different or even conflicting requirements. The most extreme example of this is perhaps scene segmentation where, in the absence of a definite goal, a concise method for the evaluation of such algorithms is likely to continue to be a challenge [26].…”