Methodology for evaluating image-segmentation algorithms

Abstract

The purpose of this paper is to describe a framework for evaluating image segmentation algorithms. Image segmentation consists of object recognition and delineation. For evaluating segmentation methods, three factors -precision (reproducibility), accuracy (agreement with truth), and efficiency (time taken) -need to be considered for both recognition and delineation. To assess precision, we need to choose a figure of merit (FOM), repeat segmentation considering all sources of variation, and determine variations in FOM via statistical analysis. It is impossible usually to establish true segmentation. Hence, to assess accuracy, we need to choose a surrogate of true segmentation and proceed as for precision. To assess efficiency, both the computational and the user time required for algorithm and operator training and for algorithm execution should be measured and analyzed. Precision, accuracy, and efficiency are interdependent. It is difficult to improve one factor without affecting others. Segmentation methods must be compared based on all three factors. The weight given to each factor depends on application.

Fast and Robust Semi-automatic Liver Segmentation with Haptic Interaction

“…As suggested in [12], a measure of the inter-operator precision η ij for two segmentation results S i and S j is the fraction of intersection and union:…”

Implementation of Computer Aided Diagnosis System Based on Parallel Approach of Ant Based Medical Image Segmentation

“…But this method did not support for additive intensity noise field. Content based model (Udupa et al, 2002) was used. However this method concentrated just to remove the baselines and this method suited to remove low frequency flat distortion noise.…”

Interactive, GPU-Based Level Sets for 3D Segmentation

“…We chose nine cases for our study: three meningioma (cases 1-3) and 6 low grade glioma (4)(5)(6)(8)(9)(10). One case, number 7, was omitted because a quick inspection showed it that its intensity structure was too complicated to be segmented by the proposed tool-such a problem remains as future work, as we will discuss in Section 5.…”

“…We consider three factors in evaluating our segmentation method [10]: validity of the results (accuracy), reproducibility of the results (precision), and efficiency of the method (time). To quantify accuracy we established a ground truth from the expert segmented slices using the STAPLE method [11].…”

“…To assess interoperator precision in segmentations, we used the metric proposed by [10], which consists of pairwise comparisons of the cardinality of the intersection of the positive classifications divided by the cardinality of the union of positive classifications. To analyze efficiency, we calculated the average total time (user time plus processing time) taken for a segmentation.…”