A formal method for selecting evaluation metrics for image segmentation

Taha, Abdel Aziz; Hanbury, Allan; Toro, Óscar Alfonso Jiménez del

doi:10.1109/icip.2014.7025187

Cited by 26 publications

(18 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, conventional HD is sensitive to and over-penalizes outliers, which are very likely in the comparison with surgical segmentations [ 47 ]. Average HD overcomes these limitations and is therefore particularly well-suited for anatomical image analyses [ 50 ]. Irrespective of the used metric, results must be still interpreted with caution—despite generally favorable global aHD, there could still be considerable local disagreement within a complex structure [ 49 ].…”

Section: Discussionmentioning

confidence: 99%

Accuracy and practical aspects of semi- and fully automatic segmentation methods for resected brain areas

et al. 2020

View full text Add to dashboard Cite

Purpose Precise segmentation of brain lesions is essential for neurological research. Specifically, resection volume estimates can aid in the assessment of residual postoperative tissue, e.g. following surgery for glioma. Furthermore, behavioral lesion-symptom mapping in epilepsy relies on accurate delineation of surgical lesions. We sought to determine whether semi- and fully automatic segmentation methods can be applied to resected brain areas and which approach provides the most accurate and cost-efficient results. Methods We compared a semi-automatic (ITK-SNAP) with a fully automatic (lesion_GNB) method for segmentation of resected brain areas in terms of accuracy with manual segmentation serving as reference. Additionally, we evaluated processing times of all three methods. We used T1w, MRI-data of epilepsy patients (n = 27; 11 m; mean age 39 years, range 16–69) who underwent temporal lobe resections (17 left). Results The semi-automatic approach yielded superior accuracy (p < 0.001) with a median Dice similarity coefficient (mDSC) of 0.78 and a median average Hausdorff distance (maHD) of 0.44 compared with the fully automatic approach (mDSC 0.58, maHD 1.32). There was no significant difference between the median percent volume difference of the two approaches (p > 0.05). Manual segmentation required more human input (30.41 min/subject) and therefore inferring significantly higher costs than semi- (3.27 min/subject) or fully automatic approaches (labor and cost approaching zero). Conclusion Semi-automatic segmentation offers the most accurate results in resected brain areas with a moderate amount of human input, thus representing a viable alternative compared with manual segmentation, especially for studies with large patient cohorts.

show abstract

Section: Discussionmentioning

confidence: 99%

Accuracy and practical aspects of semi- and fully automatic segmentation methods for resected brain areas

et al. 2020

View full text Add to dashboard Cite

show abstract

“…These are the most common used metrics for evaluating 3D medical image segmentations and include volume‐ and overlap‐based metric types . Multiple metrics are used because different metrics reflect different types of errors . For example, when segmentations are small, distance‐based metrics such as HD are recommended over overlap‐based metrics such as Dice coefficient.…”

Section: Evaluation Methodsmentioning

confidence: 99%

“…32 Multiple metrics are used because different metrics reflect different types of errors. 33 For example, when segmentations are small, distance-based metrics such as HD are recommended over overlap-based metrics such as Dice coefficient. Overlap-based metrics are recommended if volume-based statistics are important.…”

Section: A Evaluation Metricsmentioning

confidence: 99%

Evaluation of segmentation methods on head and neck CT: Auto‐segmentation challenge 2015

et al. 2017

View full text Add to dashboard Cite

Purpose: Automated delineation of structures and organs is a key step in medical imaging. However, due to the large number and diversity of structures and the large variety of segmentation algorithms, a consensus is lacking as to which automated segmentation method works best for certain applications. Segmentation challenges are a good approach for unbiased evaluation and comparison of segmentation algorithms. Methods: In this work, we describe and present the results of the Head and Neck Auto-Segmentation Challenge 2015, a satellite event at the Medical Image Computing and Computer Assisted Interventions (MICCAI) 2015 conference. Six teams participated in a challenge to segment nine structures in the head and neck region of CT images: brainstem, mandible, chiasm, bilateral optic nerves, bilateral parotid glands, and bilateral submandibular glands. Results: This paper presents the quantitative results of this challenge using multiple established error metrics and a well-defined ranking system. The strengths and weaknesses of the different auto-segmentation approaches are analyzed and discussed. Conclusions: The Head and Neck Auto-Segmentation Challenge 2015 was a good opportunity to assess the current state-of-the-art in segmentation of organs at risk for radiotherapy treatment. Participating teams had the possibility to compare their approaches to other methods under unbiased and standardized circumstances. The results demonstrate a clear tendency toward more general purpose and fewer structure-specific segmentation algorithms.

show abstract

“…No formal way is followed to choose the metrics out of many for a particular segmentation task. The most of the researchers choose the evaluation metrics according to the popularity [29]. The selection of the best evaluation metrics depends on the segmentation goal based on the task to be achieved.…”

Section: Performance Metric Evaluationmentioning

confidence: 99%

Automatic segmentation of plantar thermograms using adaptive C means technique

Prabhu

Verma

2021

IJECE

View full text Add to dashboard Cite

Diabetic foot ulcer (DFU) is one of the major concern of diabetes and it is rapidly increasing, in worst case scenario this may lead to amputation. The DFU can be avoided by the early detection and proper diagnosis. Many of the studies carried out highlights that, thermography is the most useful technique to measure the changes in the temperature of plantar surface and alerts to indicate the risk associated with DFU. The distribution of temperature does not have a fixed pattern across the patients, hence it makes the difficulty in measuring the appropriate changes. This gap will provide a scope to improve the analysis technique so as to measure the plantar surface temperature effectively and identify any abnormal changes. In this paper, the segmentation algorithm namely adaptive C means (ACM) for the image segmentation is discussed. ACM is based on the spatial information and this method includes the two stage implementation. In the first stage, nonlocal spatial information is added and in the second stage, spatial shape information is used in order to refine the constraint of local spatial. Outcome of the proposed method shows that ACM is very much effective and it outperforms the other existing methods.

show abstract

A formal method for selecting evaluation metrics for image segmentation

Cited by 26 publications

References 11 publications

Accuracy and practical aspects of semi- and fully automatic segmentation methods for resected brain areas

Accuracy and practical aspects of semi- and fully automatic segmentation methods for resected brain areas

Evaluation of segmentation methods on head and neck CT: Auto‐segmentation challenge 2015

Automatic segmentation of plantar thermograms using adaptive C means technique

Contact Info

Product

Resources

About