Uncertainty Quantification in CNN Through the Bootstrap of Convex Neural Networks

Du, Hongfei; Barut, Emre

doi:10.1609/aaai.v35i13.17434

Cited by 9 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this first version of the MPXV-CNN, predictions will also be made if the image has a low quality such as in low-light conditions or with significant blurriness. New methods like uncertainty quantifications of CNNs could help detect cases where the prediction of the MPXV-CNN should not be used 32 . Additional evaluations such as the analysis of the MPXV-CNN of multiple images from different body locations of the same patient could help to improve the performance of the MPXV-CNN.…”

Section: Discussionmentioning

confidence: 99%

A deep-learning algorithm to classify skin lesions from mpox virus infection

Thieme

Zheng

Machiraju

et al. 2023

Nat Med

View full text Add to dashboard Cite

Undetected infection and delayed isolation of infected individuals are key factors driving the monkeypox virus (now termed mpox virus or MPXV) outbreak. To enable earlier detection of MPXV infection, we developed an image-based deep convolutional neural network (named MPXV-CNN) for the identification of the characteristic skin lesions caused by MPXV. We assembled a dataset of 139,198 skin lesion images, split into training/validation and testing cohorts, comprising non-MPXV images (n = 138,522) from eight dermatological repositories and MPXV images (n = 676) from the scientific literature, news articles, social media and a prospective cohort of the Stanford University Medical Center (n = 63 images from 12 patients, all male). In the validation and testing cohorts, the sensitivity of the MPXV-CNN was 0.83 and 0.91, the specificity was 0.965 and 0.898 and the area under the curve was 0.967 and 0.966, respectively. In the prospective cohort, the sensitivity was 0.89. The classification performance of the MPXV-CNN was robust across various skin tones and body regions. To facilitate the usage of the algorithm, we developed a web-based app by which the MPXV-CNN can be accessed for patient guidance. The capability of the MPXV-CNN for identifying MPXV lesions has the potential to aid in MPXV outbreak mitigation.

show abstract

Section: Discussionmentioning

confidence: 99%

A deep-learning algorithm to classify skin lesions from mpox virus infection

Thieme

Zheng

Machiraju

et al. 2023

Nat Med

View full text Add to dashboard Cite

show abstract

“…The frequentist UQ approach has been adopted in combination with various ML models, including random forests 19,20 , boosted trees 21 , and deep neural networks [22][23][24] . As this approach is generally applicable regardless of the type of ML model, it is frequently coupled with "strong learners", i.e., the models that are capable of accurately fitting highly complex and non-stationary functions.…”

Section: Frequentist and Bayesian Uncertainty Quantification Several ...mentioning

confidence: 99%

Uncertainty-aware mixed-variable machine learning for materials design

Zhang

Iyer

Apley

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Data-driven design shows the promise of accelerating materials discovery but is challenging due to the prohibitive cost of searching the vast design space of chemistry, structure, and synthesis methods. Bayesian optimization (BO) employs uncertainty-aware machine learning models to select promising designs to evaluate, hence reducing the cost. However, BO with mixed numerical and categorical variables, which is of particular interest in materials design, has not been well studied. In this work, we survey frequentist and Bayesian approaches to uncertainty quantification of machine learning with mixed variables. We then conduct a systematic comparative study of their performances in BO using a popular representative model from each group, the random forest-based Lolo model (frequentist) and the latent variable Gaussian process model (Bayesian). We examine the efficacy of the two models in the optimization of mathematical functions, as well as properties of structural and functional materials, where we observe performance differences as related to problem dimensionality and complexity. By investigating the machine learning models’ predictive and uncertainty estimation capabilities, we provide interpretations of the observed performance differences. Our results provide practical guidance on choosing between frequentist and Bayesian uncertainty-aware machine learning models for mixed-variable BO in materials design.

show abstract

“…The frequentist UQ approach has been adopted in combination with various ML models, including random forests 22,23 , boosted trees 24 , and deep neural networks [25][26][27] . As this approach is generally applicable regardless of the type of ML model, it is frequently coupled with "strong learners", i.e., the models that are capable of accurately fitting highly complex and non-stationary functions.…”

Section: Frequentist and Bayesian Uncertainty Quantificationmentioning

confidence: 99%

Uncertainty-Aware Mixed-Variable Machine Learning for Materials Design

Zhang¹,

Iyer²,

Apley³

et al. 2022

Preprint

View full text Add to dashboard Cite

Data-driven design shows the promise of accelerating materials discovery but is challenging due to the prohibitive cost of searching the vast design space of chemistry, structure, and synthesis methods. Bayesian Optimization (BO) employs uncertainty-aware machine learning models to select promising designs to evaluate, hence reducing the cost. However, BO with mixed numerical and categorical variables, which is of particular interest in materials design, has not been well studied. In this work, we survey frequentist and Bayesian approaches to uncertainty quantification of machine learning with mixed variables. We then conduct a systematic comparative study of their performances in BO using a popular representative model from each group, the random forest-based Lolo model (frequentist) and the latent variable Gaussian process model (Bayesian). We examine the efficacy of the two models in the optimization of mathematical functions, as well as properties of structural and functional materials, where we observe performance differences as related to problem dimensionality and complexity. By investigating the machine learning models' predictive and uncertainty estimation capabilities, we provide interpretations of the observed performance differences. Our results provide practical guidance on choosing between frequentist and Bayesian uncertainty-aware machine learning models for mixed-variable BO in materials design.

show abstract

Uncertainty Quantification in CNN Through the Bootstrap of Convex Neural Networks

Cited by 9 publications

References 6 publications

A deep-learning algorithm to classify skin lesions from mpox virus infection

A deep-learning algorithm to classify skin lesions from mpox virus infection

Uncertainty-aware mixed-variable machine learning for materials design

Uncertainty-Aware Mixed-Variable Machine Learning for Materials Design

Contact Info

Product

Resources

About