Model utility of a deep learning-based segmentation is not Dice coefficient dependent: A case study in volumetric brain blood vessel segmentation

Alidoost, Mohammadali; Ghodrati, Vahid; Ahmadian, Amirhossein; Shafiee, Abbas; Hassani, Cameron H.; Bedayat, Arash; Wilson, Jennifer

doi:10.1016/j.ibmed.2023.100092

Cited by 3 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, much of the research has been on creating novel image processing algorithms leaving the critical issue of reliably and objectively evaluating the performance of these algorithms largely unexplored 50 . Moreover, some of the commonly used evaluation metrics do not always correlate with clinical applicability 47,128 , and the specific features of a biomedical problem may make certain metrics unsuitable, such as when the Dice Similarity Coefficient (DSC) is utilised to evaluate extremely small structures 129 . As a result, carefully choosing the right evaluation metric for a given problem becomes important for validating and comparing the performance of image processing methods.…”

Section: Nnu-net Frameworkmentioning

confidence: 99%

Deep Learning for 3D Vascular Segmentation in Phase Contrast Tomography

Yagis,

Aslani,

Jain

et al. 2024

Preprint

View full text Add to dashboard Cite

Automated blood vessel segmentation is critical for biomedical image analysis, as vessel morphology changes are associated with numerous pathologies. Still, precise segmentation is difficult due to the complexity of vascular structures, anatomical variations across patients, the scarcity of annotated public datasets, and the quality of images. Our goal is to provide a foundation on the topic and identify a robust baseline model for application to vascular segmentation using a new imaging modality, Hierarchical Phase-Contrast Tomography (HiP-CT). We begin with an extensive review of current machine learning approaches for vascular segmentation across various organs. Our work introduces a meticulously curated training dataset, verified by double annotators, consisting of vascular data from three kidneys imaged using Hierarchical Phase-Contrast Tomography (HiP-CT) as part of the Human Organ Atlas Project. HiP-CT, pioneered at the European Synchrotron Radiation Facility in 2020, revolutionizes 3D organ imaging by offering resolution around 20µm/voxel, and enabling highly detailed localized zooms up to 1µm/voxel without physical sectioning. We leverage the nnU-Net framework to evaluate model performance on this high-resolution dataset, using both known and novel samples, and implementing metrics tailored for vascular structures. Our comprehensive review and empirical analysis on HiP-CT data sets a new standard for evaluating machine learning models in high-resolution organ imaging. Our three experiments yielded Dice scores of 0.9523 and 0.9410, and 0.8585, respectively. Nevertheless, DSC primarily assesses voxel-to-voxel concordance, overlooking several crucial characteristics of the vessels and should not be the sole metric for deciding the performance of vascular segmentation. Our results show that while segmentations yielded reasonably high scores—such as centerline Dice values ranging from 0.82 to 0.88, certain errors persisted. Specifically, large vessels that collapsed due to the lack of hydro-static pressure (HiP-CT is an ex vivo technique) were segmented poorly. Moreover, decreased connectivity in finer vessels and higher segmentation errors at vessel boundaries were observed. Such errors, particularly in significant vessels, obstruct the understanding of the structures by interrupting vascular tree connectivity. Through our review and outputs, we aim to set a benchmark for subsequent model evaluations using various modalities, especially with the HiP-CT imaging database.

show abstract

Section: Nnu-net Frameworkmentioning

confidence: 99%

Deep Learning for 3D Vascular Segmentation in Phase Contrast Tomography

Yagis,

Aslani,

Jain

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The dice coefficient has found extensive application in MRI image segmentation [38]. This technique can be employed to assess the pixel match between the predicted segmentation and the corresponding ground truth [39]. In image segmentation, the dice coefficient is 2 times the overlap area divided by the total number of pixels in both images.…”

Section: 𝐷𝑖𝑐𝑒(𝐷 𝑄) = 2|𝐷 ∩ 𝑄| |𝐷| + |𝑄|mentioning

confidence: 99%

Evaluation of Stochastic Gradient Descent Optimizer on U-Net Architecture for Brain Tumor Segmentation

Purwono,

Mangkunegara

2023

IJRCS

View full text Add to dashboard Cite

A brain tumor is a type of disease that is quite dangerous in the world. This disease is one of the main causes of human death and has a high risk of recurrence. There are several types of brain tumor locations such as edema, necrosis to elevation. Segmenting the location of this disease is important to do to support faster recovery efforts. The Convolutional Neural Network (CNN) algorithm, which is part of the deep learning method, can be an alternative to this segmentation effort. The U-Net architecture is part of the CNN algorithm which specifically works on medical image segmentation. This study experimented to build a special U-Net architecture for medical image segmentation that had been optimized with SGD. The data used is BraTS2020O which contains a collection of MRI data. This optimization aims to improve the performance of the U-net architecture for segmenting brain tumor images. The results of the study show that the SGD optimization carried out has succeeded in providing better performance than previous studies. This can be seen from the performance value obtained at 0.9879. This accuracy value indicates an increase in accuracy from previous studies. High accuracy indicates that the SGD-optimized model has good segmentation prediction performance.

show abstract

“…Emphasizing an understanding of drug‐induced effects instead of overall performance is consistent with other computational efforts that emphasize model utility over performance. This is a growing field of computational research that we 28 and others have emphasized 29 …”

Section: Introductionmentioning

confidence: 99%

Preclinical side effect prediction through pathway engineering of protein interaction network models

Alidoost,

Wilson

2024

CPT Pharmacom & Syst Pharma

Self Cite

View full text Add to dashboard Cite

Modeling tools aim to predict potential drug side effects, although they suffer from imperfect performance. Specifically, protein–protein interaction models predict drug effects from proteins surrounding drug targets, but they tend to overpredict drug phenotypes and require well‐defined pathway phenotypes. In this study, we used PathFX, a protein–protein interaction tool, to predict side effects for active ingredient‐side effect pairs extracted from drug labels. We observed limited performance and defined new pathway phenotypes using pathway engineering strategies. We defined new pathway phenotypes using a network‐based and gene expression‐based approach. Overall, we discovered a trade‐off between sensitivity and specificity values and demonstrated a way to limit overprediction for side effects with sufficient true positive examples. We compared our predictions to animal models and demonstrated similar performance metrics, suggesting that protein–protein interaction models do not need perfect evaluation metrics to be useful. Pathway engineering, through the inclusion of true positive examples and omics measurements, emerges as a promising approach to enhance the utility of protein interaction network models for drug effect prediction.

show abstract

Model utility of a deep learning-based segmentation is not Dice coefficient dependent: A case study in volumetric brain blood vessel segmentation

Cited by 3 publications

References 18 publications

Deep Learning for 3D Vascular Segmentation in Phase Contrast Tomography

Deep Learning for 3D Vascular Segmentation in Phase Contrast Tomography

Evaluation of Stochastic Gradient Descent Optimizer on U-Net Architecture for Brain Tumor Segmentation

Preclinical side effect prediction through pathway engineering of protein interaction network models

Contact Info

Product

Resources

About