Quantitative Comparison of Monte-Carlo Dropout Uncertainty Measures for Multi-class Segmentation

Camarasa, Robin; Bos, Daniel; Hendrikse, Jeroen; Nederkoorn, Paul H. J.; Kooi, M. Eline; Lugt, Aad van der; Bruijne, Marleen de

doi:10.1007/978-3-030-60365-6_4

Cited by 28 publications

(20 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(1) MCD-x,max: In a majority of methods using MCD (Leibig et al, 2017;Roy et al, 2018;Camarasa et al, 2020;Kwon et al, 2020;Nair et al, 2020), the prediction and the uncertainty are computed as the mean and the variance of multiple forward predictions, respectively. Following these approaches, first, the maximum of each heatmap ĥ(k)…”

Section: Methodsmentioning

confidence: 99%

“…Uncertainty estimation in medical imaging has been widely adopted and used in segmentation, e.g., for segmenting multiple sclerosis lesions (Nair et al, 2020), the whole-brain (Roy et al, 2018), the fetal brain and brain tumors (Wang et al, 2019a), the carotid artery (Camarasa et al, 2020) as well as for ischemic stroke lesion and vessel extraction from retinal images (Kwon et al, 2020). Furthermore, uncertainty estimation was applied for disease detection using a diabetic retinopathy dataset (Leibig et al, 2017;Ayhan and Berens, 2018), for classification of skin lesions (Mobiny et al, 2019) and for probabilistic classifier-based image registration (Sedghi et al, 2019).…”

Section: Uncertainty Estimationmentioning

confidence: 99%

“…Here, the weight distributions are approximated by reusing Dropout (Srivastava et al, 2014) during inference to retrieve a set of stochastic predictions. Since MCD is easy to implement and computationally more efficient than BNNs, it is the most common method to model epistemic uncertainty in medical imaging (Leibig et al, 2017;Roy et al, 2018;Mobiny et al, 2019;Wang et al, 2019a;Camarasa et al, 2020;Kwon et al, 2020;Nair et al, 2020).…”

Section: Uncertainty Estimationmentioning

confidence: 99%

See 2 more Smart Citations

Modeling Annotation Uncertainty with Gaussian Heatmaps in Landmark Localization

Thaler¹,

Payer²,

Urschler³

et al. 2021

Preprint

View full text Add to dashboard Cite

In landmark localization, due to ambiguities in defining their exact position, landmark annotations may suffer from large observer variabilities, which result in uncertain annotations. To model the annotation ambiguities of the training dataset, we propose to learn anisotropic Gaussian parameters modeling the shape of the target heatmap during optimization. Furthermore, our method models the prediction uncertainty of individual samples by fitting anisotropic Gaussian functions to the predicted heatmaps during inference. Besides state-of-the-art results, our experiments on datasets of hand radiographs and lateral cephalograms also show that Gaussian functions are correlated with both localization accuracy and observer variability. As a final experiment, we show the importance of integrating the uncertainty into decision making by measuring the influence of the predicted location uncertainty on the classification of anatomical abnormalities in lateral cephalograms.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Uncertainty Estimationmentioning

confidence: 99%

Section: Uncertainty Estimationmentioning

confidence: 99%

See 1 more Smart Citation

Modeling Annotation Uncertainty with Gaussian Heatmaps in Landmark Localization

Thaler¹,

Payer²,

Urschler³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Gal et al [14] proposed to enable dropout at test time as a Bayesian approximation to sample multiple different predictions. From these Monte Carlo (MC) predictions, it is possible to derive uncertainty metrics that are indicative of model performance [15] which has already been explored for multiple medical image classification tasks [16][17][18]. The final prediction is usually generated by taking the average over all MC predictions.…”

Section: Dropoutmentioning

confidence: 99%

Improving the repeatability of deep learning models with Monte Carlo dropout

Lemay¹,

Hoebel²,

Bridge³

et al. 2022

Preprint

View full text Add to dashboard Cite

The integration of artificial intelligence into clinical workflows requires reliable and robust models. Repeatability is a key attribute of model robustness. Repeatable models output predictions with low variation during independent tests carried out under similar conditions. During model development and evaluation, much attention is given to classification performance while model repeatability is rarely assessed, leading to

show abstract

“…But it has been found [53] that if the initial pseudo-label estimates are erroneous, then using them directly in the segmentation loss function can lead to possible degradation of performance. To avoid this, some methods [54], [55], [56], [57], [58], [59], [60] integrate uncertainty or confidence estimates [61], [62], [63] of pseudo-labels into self-training to control the quality of pseudo-labels used for training and thereby reduce the negative effects of poor quality of pseudo-labels.…”

Section: Related Workmentioning

confidence: 99%

Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation

Chaitanya¹,

Erdil²,

Karani³

et al. 2021

Preprint

View full text Add to dashboard Cite

Supervised deep learning-based methods yield accurate results for medical image segmentation. However, they require large labeled datasets for this, and obtaining them is a laborious task that requires clinical expertise. Semi/selfsupervised learning-based approaches address this limitation by exploiting unlabeled data along with limited annotated data. Recent self-supervised learning methods use contrastive loss to learn good global level representations from unlabeled images and achieve high performance in classification tasks on popular natural image datasets like ImageNet. In pixel-level prediction tasks such as segmentation, it is crucial to also learn good local level representations along with global representations to achieve better accuracy. However, the impact of the existing local contrastive loss-based methods remains limited for learning good local representations because similar and dissimilar local regions are defined based on random augmentations and spatial proximity; not based on the semantic label of local regions due to lack of large-scale expert annotations in the semi/self-supervised setting. In this paper, we propose a local contrastive loss to learn good pixel level features useful for segmentation by exploiting semantic label information obtained from pseudo-labels of unlabeled images alongside limited annotated images. In particular, we define the proposed loss to encourage similar representations for the pixels that have the same pseudo-label/ label while being dissimilar to the representation of pixels with different pseudo-label/label in the dataset. We perform pseudo-label based self-training and train the network by jointly optimizing the proposed contrastive loss on both labeled and unlabeled sets and segmentation loss on only the limited labeled set. We evaluated the proposed approach on three public medical datasets of cardiac and prostate anatomies, and obtain high segmentation performance with a limited labeled set of one or two 3D volumes. Extensive comparisons with the state-of-the-art semi-supervised and data augmentation methods and concurrent contrastive learning methods demonstrate the substantial improvement achieved by the proposed method. The code is made publicly available at https://github.com/krishnabits001/pseudo_label_contrastive_training.

show abstract

Quantitative Comparison of Monte-Carlo Dropout Uncertainty Measures for Multi-class Segmentation

Cited by 28 publications

References 35 publications

Modeling Annotation Uncertainty with Gaussian Heatmaps in Landmark Localization

Modeling Annotation Uncertainty with Gaussian Heatmaps in Landmark Localization

Improving the repeatability of deep learning models with Monte Carlo dropout

Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation

Contact Info

Product

Resources

About