Estimation of small failure probabilities based on thermodynamic integration and parallel tempering

Xiao, Sun; Reuschen, Sebastian; Köse, Gözde Güsan; Oladyshkin, Sergey; Nowak, Wolfgang

doi:10.1016/j.ymssp.2019.106248

Cited by 20 publications

(4 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, considering alternative approaches for the intermediate steps in both phases could be interesting, such as incorporating a method based on smoothed indicator functions and thermodynamic integration proposed by Xiao et al. (2019), in the second phase. Finally, exploring test cases that do not rely on Gaussian assumptions would be intriguing, as previously undertaken for the posterior part in Amaya et al.…”

Section: Discussionmentioning

confidence: 99%

“…Surrogates (e.g., Razavi et al, 2012) in this context can serve as simplified models or approximations of the underlying system, allowing for faster evaluations and reducing the computational burden. Moreover, considering alternative approaches for the intermediate steps in both phases could be interesting, such as incorporating a method based on smoothed indicator functions and thermodynamic integration proposed by Xiao et al (2019), in the second phase. Finally, exploring test cases that do not rely on Gaussian assumptions would be intriguing, as previously undertaken for the posterior part in Amaya et al (2021Amaya et al ( , 2022.…”

Section: Water Resources Researchmentioning

confidence: 99%

See 1 more Smart Citation

Rare Event Probability Estimation for Groundwater Inverse Problems With a Two‐Stage Sequential Monte Carlo Approach

Friedli,

Linde

2024

Water Resources Research

View full text Add to dashboard Cite

Bayesian inversions followed by estimations of rare event probabilities are often needed to analyze groundwater hazards. Instead of focusing on the posterior distribution of model parameters, the main interest lies then in the distribution of a specific quantity of interest contingent upon these parameters. To address the associated methodological challenges, we introduce a two‐stage Sequential Monte Carlo approach. In the first stage, it generates particles that approximate the posterior distribution; in the second stage, it employs subset sampling techniques to assess the probability of the rare event of interest. By considering two hydrogeological problems of increasing complexity, we showcase the efficiency and accuracy of the resulting PostRisk‐SMC method for rare event probability estimation related to groundwater hazards. We compare the performance of the PostRisk‐SMC method with a traditional Monte Carlo approach that relies on Markov chain Monte Carlo samples. We showcase that our estimates align with those of the traditional method, but the coefficients of variation are notably lower for the same computational budget when targeting more rare events. Furthermore, we highlight that the PostRisk‐SMC method allows estimating rare event probabilities approaching one in a billion using less than one hundred thousand forward simulations. Even if the presented examples are related to groundwater hazards, the methodology is well‐suited for addressing a wide range of topics in the geosciences and beyond.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Water Resources Researchmentioning

confidence: 99%

Rare Event Probability Estimation for Groundwater Inverse Problems With a Two‐Stage Sequential Monte Carlo Approach

Friedli,

Linde

2024

Water Resources Research

View full text Add to dashboard Cite

show abstract

“…Direct implementation of Bayesian principles for the original physical model is usually not feasible using Monte Carlo (MC) simulations [ 26 ] or even Markov chain Monte Carlo (MCMC) approaches [ 27 ]. Any advanced technique, such as thermodynamic integration [ 28 ], parallel tempering [ 29 ], nested sampling [ 30 , 31 ], subset simulation [ 32 , 33 ], or Gaussian mixture importance sampling [ 34 ] is still not feasible for applications where the original model is very expensive.…”

Section: Introductionmentioning

confidence: 99%

Bayesian3 Active Learning for the Gaussian Process Emulator Using Information Theory

Oladyshkin

Mohammadi

Kroeker

et al. 2020

Entropy

Self Cite

View full text Add to dashboard Cite

Gaussian process emulators (GPE) are a machine learning approach that replicates computational demanding models using training runs of that model. Constructing such a surrogate is very challenging and, in the context of Bayesian inference, the training runs should be well invested. The current paper offers a fully Bayesian view on GPEs for Bayesian inference accompanied by Bayesian active learning (BAL). We introduce three BAL strategies that adaptively identify training sets for the GPE using information-theoretic arguments. The first strategy relies on Bayesian model evidence that indicates the GPE’s quality of matching the measurement data, the second strategy is based on relative entropy that indicates the relative information gain for the GPE, and the third is founded on information entropy that indicates the missing information in the GPE. We illustrate the performance of our three strategies using analytical- and carbon-dioxide benchmarks. The paper shows evidence of convergence against a reference solution and demonstrates quantification of post-calibration uncertainty by comparing the introduced three strategies. We conclude that Bayesian model evidence-based and relative entropy-based strategies outperform the entropy-based strategy because the latter can be misleading during the BAL. The relative entropy-based strategy demonstrates superior performance to the Bayesian model evidence-based strategy.

show abstract

“…It is well known that such plain Monte Carlo techniques require a large number of model runs, and become computationally very demanding for many applied problems [6,9]. Thermodynamic integration [10], thermodynamic integration combined with parallel tempering [11], nested sampling [12,13], Gaussian mixture importance sampling [14] or employment of surrogates [15] were proposed in the literature to reduce the computational burden of estimating BME. However, surrogates include approximation errors due to the reduced models, so that estimated BME values should incorporate a correction factor that helps to assure a reliable model ranking especially under strong computational time constraints [16].…”

Section: Introductionmentioning

confidence: 99%

The Connection between Bayesian Inference and Information Theory for Model Selection, Information Gain and Experimental Design

Oladyshkin

Nowak

2019

Entropy

Self Cite

View full text Add to dashboard Cite

We show a link between Bayesian inference and information theory that is useful for model selection, assessment of information entropy and experimental design. We align Bayesian model evidence (BME) with relative entropy and cross entropy in order to simplify computations using prior-based (Monte Carlo) or posterior-based (Markov chain Monte Carlo) BME estimates. On the one hand, we demonstrate how Bayesian model selection can profit from information theory to estimate BME values via posterior-based techniques. Hence, we use various assumptions including relations to several information criteria. On the other hand, we demonstrate how relative entropy can profit from BME to assess information entropy during Bayesian updating and to assess utility in Bayesian experimental design. Specifically, we emphasize that relative entropy can be computed avoiding unnecessary multidimensional integration from both prior and posterior-based sampling techniques. Prior-based computation does not require any assumptions, however posterior-based estimates require at least one assumption. We illustrate the performance of the discussed estimates of BME, information entropy and experiment utility using a transparent, non-linear example. The multivariate Gaussian posterior estimate includes least assumptions and shows the best performance for BME estimation, information entropy and experiment utility from posterior-based sampling.

show abstract

Estimation of small failure probabilities based on thermodynamic integration and parallel tempering

Cited by 20 publications

References 51 publications

Rare Event Probability Estimation for Groundwater Inverse Problems With a Two‐Stage Sequential Monte Carlo Approach

Rare Event Probability Estimation for Groundwater Inverse Problems With a Two‐Stage Sequential Monte Carlo Approach

Bayesian3 Active Learning for the Gaussian Process Emulator Using Information Theory

The Connection between Bayesian Inference and Information Theory for Model Selection, Information Gain and Experimental Design

Contact Info

Product

Resources

About