Hessian-based adaptive sparse quadrature for infinite-dimensional Bayesian inverse problems

Chen, Peng; Villa, Umberto; Ghattas, Omar

doi:10.1016/j.cma.2017.08.016

Cited by 51 publications

(45 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the resulting posterior measure. For example, the computational burden of expensive forward or likelihood models can be reduced by surrogates or multilevel strategies [14,20,27,34] and for many classical sampling or integration methods such as Quasi-Monte Carlo [12], Markov chain Monte Carlo [6,32,42], and numerical quadrature [5,35] we now know modifications and conditions which ensure a dimension-independent efficiency.…”

Section: Introductionmentioning

confidence: 99%

On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems

2020

View full text Add to dashboard Cite

The Bayesian approach to inverse problems provides a rigorous framework for the incorporation and quantification of uncertainties in measurements, parameters and models. We are interested in designing numerical methods which are robust w.r.t. the size of the observational noise, i.e., methods which behave well in case of concentrated posterior measures. The concentration of the posterior is a highly desirable situation in practice, since it relates to informative or large data. However, it can pose a computational challenge for numerical methods based on the prior measure. We propose to employ the Laplace approximation of the posterior as the base measure for numerical integration in this context. The Laplace approximation is a Gaussian measure centered at the maximum a-posteriori estimate and with covariance matrix depending on the logposterior density. We discuss convergence results of the Laplace approximation in terms of the Hellinger distance and analyze the efficiency of Monte Carlo methods based on it. In particular, we show that Laplace-based importance sampling and Laplace-based quasi-Monte-Carlo methods are robust w.r.t. the concentration of the posterior for large classes of posterior distributions and integrands whereas prior-based importance sampling and plain quasi-Monte Carlo are not. Numerical experiments are presented to illustrate the theoretical findings.

show abstract

Section: Introductionmentioning

confidence: 99%

On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems

2020

View full text Add to dashboard Cite

show abstract

“…In case of the current version of the BayesFactor package, such an option is an alternative algorithm using Laplace approximation. Because this optional algorithm does not use any sampling, it will provide stable results across runs (Schillings, Sprungk, & Wacker, 2020; for alternative algorithms with stable, analytic results, see Chen, Villa, & Ghattas, 2017;Schillings & Schwab, 2013). At the same time, Laplace approximation can return systematically different results than (many iterations of) MCMC-based methods, especially when sample sizes are small.…”

Section: Tablementioning

confidence: 99%

Variability of Bayes Factor estimates in Bayesian Analysis of Variance

Pfister¹

2021

TQMP

View full text Add to dashboard Cite

Bayes Factor estimation for Bayesian Analysis of Variance (ANOVA) typically relies on iterative algorithms that, by design, yield slightly different results on every run of the analysis. The variability of these estimates is surprisingly large, however: The present simulations indicate that repeating one and the same Bayesian ANOVA on a constant dataset often results in Bayes Factors that differ by a factor of 2 or more within only a few runs when using common analysis procedures. Results may at times even suggest evidence for the null hypothesis of no effect on one run while supporting the alternative hypothesis on another run. These observations call for a cautious approach to the results of Bayesian ANOVAs at present, and I outline three possibilities to circumvent or minimize this limitation.

show abstract

“…Improved bounds are demonstrated in [66] and show that the Gaussian estimator becomes very inefficient if the stable rank of the operator is small and that it allows for small N only if the eigenvalues are all of approximately the same size. This means that the randomized Gaussian estimator is not a viable solution to estimate the trace of H, since it has been observed numerically or proven analytically that for many problems the Hessian operator is either nearly low-rank or its eigenvalues exhibit fast decay [8,36,19,21,20,18,23,30,2,3,1,33,64,46,58,22,24].…”

Section: Computation Of the Gradient And Hessian Of The Control Objecmentioning

confidence: 99%

Taylor approximation and variance reduction for PDE-constrained optimal control under uncertainty

Chen

Villa

Ghattas

2019

Journal of Computational Physics

Self Cite

View full text Add to dashboard Cite

In this work we develop a scalable computational framework for the solution of PDE-constrained optimal control under high-dimensional uncertainty. Specifically, we consider a mean-variance formulation of the control objective and employ a Taylor expansion with respect to the uncertain parameter either to directly approximate the control objective or as a control variate for variance reduction. The expressions for the mean and variance of the Taylor approximation are known analytically, although their evaluation requires efficient computation of the trace of the (preconditioned) Hessian of the control objective. We propose to estimate this trace by solving a generalized eigenvalue problem using a randomized algorithm that only requires the action of the Hessian on a small number of random directions. Then, the computational work does not depend on the nominal dimension of the uncertain parameter, but depends only on the effective dimension (i.e., the rank of the preconditioned Hessian), thus ensuring scalability to high-dimensional problems. Moreover, to increase the estimation accuracy of the mean and variance of the control objective by the Taylor approximation, we use it as a control variate for variance reduction, which results in considerable computational savings (several orders of magnitude) compared to a plain Monte Carlo method. In summary, our approach amounts to solving an optimal control constrained by the original PDE and a set of linearized PDEs, which arise from the computation of the gradient and Hessian of the control objective with respect to the uncertain parameter. We use the Lagrangian formalism to derive expressions for the gradient with respect to the control and apply a gradient-based optimization method to solve the problem. We demonstrate the accuracy, efficiency, and scalability of the proposed computational method for two examples with high-dimensional uncertain parameters: subsurface flow in a porous medium modeled as an elliptic PDE, and turbulent jet flow modeled by the Reynolds-averaged Navier-Stokes equations coupled with a nonlinear advection-diffusion equation characterizing model uncertainty. In particular, for the latter more challenging example we show scalability of our algorithm up to one million parameters resulting from discretization of the uncertain parameter field.

show abstract

Hessian-based adaptive sparse quadrature for infinite-dimensional Bayesian inverse problems

Cited by 51 publications

References 21 publications

On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems

On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems

Variability of Bayes Factor estimates in Bayesian Analysis of Variance

Taylor approximation and variance reduction for PDE-constrained optimal control under uncertainty

Contact Info

Product

Resources

About