Uncertainty Estimation via Stochastic Batch Normalization

Atanov, Andrei; Ashukha, Arsenii; Molchanov, Dmitry; Neklyudov, Kirill; Vetrov, Dmitry

doi:10.48550/arxiv.1802.04893

Cited by 5 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…15. In another research, Atanov et al [165] introduced a probabilistic model and showed that Batch Normalization (BN) approach can maximize the lower bound of its related marginalized log-likelihood. Since inference computationally was not efficient, they proposed the Stochastic BN (SBN) approach for approximation of appropriate inference procedure, as an uncertainty estimation method.…”

Section: Deep Gaussian Processesmentioning

confidence: 99%

A review of uncertainty quantification in deep learning: Techniques, applications and challenges

et al. 2021

View full text Add to dashboard Cite

Section: Deep Gaussian Processesmentioning

confidence: 99%

A review of uncertainty quantification in deep learning: Techniques, applications and challenges

et al. 2021

View full text Add to dashboard Cite

“…The uncertainty caused by the parameters in a neural network is known as epistemic uncertainty, which is modeled by placing a prior distribution (e.g., a Gaussian prior distribution: W ∼ N(0, I)) on the parameters of a network and then attempting to capture how much these weights vary given specific data. Recent efforts in this area include the Bayes by Backprop (Blundell et al 2015), its closely related mean-field variational inference by assuming a Gaussian prior distribution (Tölle et al 2021), stochastic batch normalization (Atanov et al 2018), and Monte-Carlo (MC) dropout (Gal and Ghahramani 2016;Kendall and Gal 2017). The applications of Bayesian deep learning in medical imaging expands on image denoising (Tölle et al 2021;Laves et al 2020b) and image segmentation (DeVries and Taylor 2018; Baumgartner et al 2019;Mehrtash et al 2020).…”

Section: Bayesian Deep Learningmentioning

confidence: 99%

TransMorph: Transformer for unsupervised medical image registration

Chen

et al. 2021

Preprint

View full text Add to dashboard Cite

In the last decade, convolutional neural networks (ConvNets) have dominated the field of medical image analysis. However, it is found that the performances of ConvNets may still be limited by their inability to model long-range spatial relations between voxels in an image. Numerous vision Transformers have been proposed recently to address the shortcomings of ConvNets, demonstrating state-of-the-art performances in many medical imaging applications. Transformers may be a strong candidate for image registration because their self-attention mechanism enables a more precise comprehension of the spatial correspondence between moving and fixed images. In this paper, we present TransMorph, a hybrid Transformer-ConvNet model for volumetric medical image registration. We also introduce three variants of TransMorph, with two diffeomorphic variants ensuring the topology-preserving deformations and a Bayesian variant producing a well-calibrated registration uncertainty estimate. The proposed models are extensively validated against a variety of existing registration methods and Transformer architectures using volumetric medical images from two applications: inter-patient brain MRI registration and phantom-to-CT registration. Qualitative and quantitative results demonstrate that TransMorph and its variants lead to a substantial performance improvement over the baseline methods, demonstrating the effectiveness of Transformers for medical image registration.

show abstract

“…Various approximations of BNNs exist that estimate the true posterior p(w|F, Y ), with a variational distribution q(w), which is feasible to train. Popular BNN approximations include stochastic variational inference [52]- [54], multiplicative normalizing flows [55], stochastic batch normalization [56], and variational inference by Monte-Carlo (MC) dropout [57], [58]. Also, non-Bayesian approaches generally train many deep ensemble or bootstrap models [59], [60] to measure the uncertainty through predictive variance.…”

Section: E Epistemic Uncertaintymentioning

confidence: 99%

U-LanD: Uncertainty-Driven Video Landmark Detection

Jafari¹,

Luong²,

Tsang³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper presents U-LanD, a framework for joint detection of key frames and landmarks in videos. We tackle a specifically challenging problem, where training labels are noisy and highly sparse. U-LanD builds upon a pivotal observation: a deep Bayesian landmark detector solely trained on key video frames, has significantly lower predictive uncertainty on those frames vs. other frames in videos. We use this observation as an unsupervised signal to automatically recognize key frames on which we detect landmarks. As a test-bed for our framework, we use ultrasound imaging videos of the heart, where sparse and noisy clinical labels are only available for a single frame in each video. Using data from 4,493 patients, we demonstrate that U-LanD can exceedingly outperform the state-of-the-art non-Bayesian counterpart by a noticeable absolute margin of 42% in R 2 score, with almost no overhead imposed on the model size. Our approach is generic and can be potentially applied to other challenging data with noisy and sparse training labels.

show abstract

Uncertainty Estimation via Stochastic Batch Normalization

Cited by 5 publications

References 0 publications

A review of uncertainty quantification in deep learning: Techniques, applications and challenges

A review of uncertainty quantification in deep learning: Techniques, applications and challenges

TransMorph: Transformer for unsupervised medical image registration

U-LanD: Uncertainty-Driven Video Landmark Detection

Contact Info

Product

Resources

About