Uncertainty Quantification for Machine Learning

Stracuzzi, David John; Chen, Maximillian; Darling, Michael Christopher; Peterson, Matthew Gregor; Vollmer, Charlie

doi:10.2172/1733262

Cited by 3 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Resampling data to estimate errors has a long history in statistics through methods such as bootstrapping (Efron & Tibshirani 1993) and has also been shown to reliably produce error estimates when applied to ML algorithms (Stracuzzi et al 2017).…”

Section: Aggregation Of Predictionsmentioning

confidence: 99%

A Comparative Study of Machine-learning Methods for X-Ray Binary Classification

Beurs

Islam

Gopalan

et al. 2022

ApJ

View full text Add to dashboard Cite

X-ray binaries (XRBs) consist of a compact object that accretes material from an orbiting secondary star. The most secure method we have for determining if the compact object is a black hole is to determine its mass: This is limited to bright objects and requires substantial time-intensive spectroscopic monitoring. With new X-ray sources being discovered with different X-ray observatories, developing efficient, robust means to classify compact objects becomes increasingly important. We compare three machine-learning classification methods (Bayesian Gaussian Processes (BGPs), K-Nearest Neighbors (KNN), Support Vector Machines) for determining whether the compact objects are neutron stars or black holes (BHs) in XRB systems. Each machine-learning method uses spatial patterns that exist between systems of the same type in 3D color–color–intensity diagrams. We used lightcurves extracted using 6 yr of data with MAXI/GSC for 44 representative sources. We find that all three methods are highly accurate in distinguishing pulsing from nonpulsing neutron stars (NPNS) with 95% of NPNS and 100% of pulsars accurately predicted. All three methods have high accuracy in distinguishing BHs from pulsars (92%) but continue to confuse BHs with a subclass of NPNS, called bursters, with KNN doing the best at only 50% accuracy for predicting BHs. The precision of all three methods is high, providing equivalent results over 5–10 independent runs. In future work, we will suggest a fourth dimension be incorporated to mitigate the confusion of BHs with bursters. This work paves the way toward more robust methods to efficiently distinguish BHs, NPNS, and pulsars.

show abstract

Section: Aggregation Of Predictionsmentioning

confidence: 99%

A Comparative Study of Machine-learning Methods for X-Ray Binary Classification

Beurs

Islam

Gopalan

et al. 2022

ApJ

View full text Add to dashboard Cite

show abstract

Uncertainty assessment in unsupervised machine-learning methods for deepwater channel seismic facies using outcrop-derived 3D models and synthetic seismic data

La Marca,

Bedle,

Stright

et al. 2024

Interpretation

View full text Add to dashboard Cite

Unsupervised machine learning (ML) techniques have been widely applied to analyze seismic reflection data, including the identification of seismic facies and structural features. However, interpreting the resulting clusters often relies on geoscientists’ expertise, necessitating a robustness assessment of these methods. To evaluate their reliability, synthetic data generated from an actual outcrop model were employed to demonstrate how two unsupervised methods, Self-Organizing Maps (SOM) and Generative Topographic Maps (GTM), cluster deepwater channel-related seismic facies and then measure the associated error. Six seismic attributes, comprising RMS amplitude, instantaneous envelope, peak magnitude, and spectral decomposition frequencies at 20, 40, and 55 Hz, served as input variables. Geobodies were assigned to each cluster formed, and error in facies clustering was quantified by comparing the actual 3D model with the facies grouped by machine learning methods on a voxel-by-voxel basis. This allowed for error quantification and the computation of metrics such as F1 score and accuracy through correlation matrices. Key findings revealed that (1) GTM and SOM exhibited similar performance, with a clustering configuration of 81 for GTM slightly outperforming others. (2) Error rates were approximately 2% for the predominant facies (background shale) but significantly higher for individual channel-related facies, suggesting that channel clusters might represent multiple facies. (3) Resolution and imbalanced data distribution impacted seismic facies predictability, resulting in nonuniqueness in cluster generation. (4) Using synthetic seismic data proved valuable for experimenting with different unsupervised ML, highlighting the need for assessing uncertainty in these methods, given their implications for crucial economic decisions reliant on reservoir interpretation, modeling, and volumetric estimations.

show abstract

In Silico Trial Approach for Biomedical Products: A Regulatory Perspective

Jose

Shifali

Mathew

et al. 2022

CCHTS

View full text Add to dashboard Cite

The modern pharmaceutical industry is creating a transition from traditional methods to advanced technologies like artificial intelligence. In the current scenario, continuous efforts are being made to incorporate computational modelling and simulation in drug discovery, development, design, and optimization. With the advancement in technology and modernization, many pharmaceutical companies are approaching in silico trials to develop safe and efficacious medicinal products. To obtain marketing authorization for a medicinal product from the concerned National regulatory Authority, manufacturers must provide evidence for the safety, efficacy, and quality of medical products in the form of in vitro or in vivo methods. However, more recently this evidence was provided to regulatory agencies in the form of modelling and simulation, i.e., in silico evidence. Such evidence (computational or experimental) will only be accepted by the regulatory authorities if it considered as qualified by them and this will require the assessment of the overall credibility of the method. One must consider the scrutiny provided by the regulatory authority to develop or use the new in silico evidence. The United States Food and Drug Administration and European Medicines Agency are the two regulatory agencies in the world that accept and encourage the use of modelling and simulation within the regulatory process. More efforts must be made by other regulatory agencies worldwide to incorporate such new evidence, i.e., modelling and simulation (in silico) within the regulatory process. This review article focuses on the approaches of in silico trials, its verification, validation, and uncertainty quantification involved in the regulatory evaluation of biomedical products that utilize predictive models.

show abstract

Uncertainty Quantification for Machine Learning

Cited by 3 publications

References 0 publications

A Comparative Study of Machine-learning Methods for X-Ray Binary Classification

A Comparative Study of Machine-learning Methods for X-Ray Binary Classification

Uncertainty assessment in unsupervised machine-learning methods for deepwater channel seismic facies using outcrop-derived 3D models and synthetic seismic data

In Silico Trial Approach for Biomedical Products: A Regulatory Perspective

Contact Info

Product

Resources

About