Multilevel Adaptive Sparse Leja Approximations for Bayesian Inverse Problems

Abstract

Deterministic interpolation and quadrature methods are often unsuitable to address Bayesian inverse problems depending on computationally expensive forward mathematical models. While interpolation may give precise posterior approximations, deterministic quadrature is usually unable to efficiently investigate an informative and thus concentrated likelihood. This leads to a large number of required expensive evaluations of the mathematical model. To overcome these challenges, we formulate and test a multilevel adaptive sparse Leja algorithm. At each level, adaptive sparse grid interpolation and quadrature are used to approximate the posterior and perform all quadrature operations, respectively. Specifically, our algorithm uses coarse discretizations of the underlying mathematical model to investigate the parameter space and to identify areas of high posterior probability. Adaptive sparse grid algorithms are then used to place points in these areas, and ignore other areas of small posterior probability. The points are weighted Leja points. As the model discretization is coarse, the construction of the sparse grid is computationally efficient. On this sparse grid, the posterior measure can be approximated accurately with few expensive, fine model discretizations. The efficiency of the algorithm can be enhanced further by exploiting more than two discretization levels. We apply the proposed multilevel adaptive sparse Leja algorithm in numerical experiments involving elliptic inverse problems in 2D and 3D space, in which we compare it with Markov chain Monte Carlo sampling and a standard multilevel approximation.

An hp‐adaptive multi‐element stochastic collocation method for surrogate modeling with information re‐use

“…However, in the large data—small noise regime, the likelihood function is highly concentrated and global surrogate modeling is difficult. Adaptive 38 and possibly multilevel 39 approaches are a popular remedy in this case. Another possibility is to use the spectral stochastic embedding (SSE) method 37 .…”

“…With respect to interpolation in particular, the Lebesgue constant of Leja sequence based interpolation grids is known to grow subexponentially, 47‐49 thus resulting in stable interpolations. Additionally, interpolation and quadrature grids with respect to any continuous PDF can be constructed by using weighted Leja sequences 39,50‐52 . Moreover, due to the fact that Leja sequences are by definition nested, that is, $$$ {\left\{{x}_i\right\}}_{i=0}^j\subset {\left\{{x}_i\right\}}_{i=0}^{j+1} $$$, they allow for re‐using readily available Leja points and model evaluations on those points in case the sequence is further expanded.…”

“…Additionally, interpolation and quadrature grids with respect to any continuous PDF can be constructed by using weighted Leja sequences. 39,[50][51][52] Moreover, due to the fact that Leja sequences are by definition nested, that is, {x i } j i=0 ⊂ {x i } j+1 i=0 , they allow for re-using readily available Leja points and model evaluations on those points in case the sequence is further expanded. Due to the nestedness property, Leja points are natural candidates for constructing sparse interpolation or quadrature grids.…”

Variational Inference for Nonlinear Inverse Problems via Neural Net Kernels: Comparison to Bayesian Neural Networks, Application to Topology Optimization

“…In a similar vein, authors in [19][20][21][22][23][24][25][26][27][28][29] have tackled the issue of efficient MCMC sampling by developing methods on Langevin and Hamiltonian MCMC, dimensionality reduction MCMC and randomized/optimized MCMC, etc. The issue of large-scale forward computation, as an indispensable part of inverse problems is addressed in [8,12,25,[30][31][32][33][33][34][35][36][37][38][39][40][41][42] via polynomial approximation, model reduction (greedy reduced basis) and multifidelity/multilevel modeling. It is also worth mentioning that part of our approach which relies on optimization of forward model can benefit from mutifidelity approximation to accelerate the forward model and adjoint computation [43][44][45][46][47].…”

An $hp$-adaptive multi-element stochastic collocation method for surrogate modeling with information re-use

“…With respect to interpolation in particular, the Lebesgue constant of Leja sequence based interpolation grids is known to grow subexponentially [39][40][41] , thus resulting in comparatively stable interpolations. Additionally, interpolation and quadrature grids with respect to any continuous PDF can be constructed by using weighted Leja sequences [42][43][44][45] . Moreover, due to the fact that Leja sequences are by definition nested, i.e.…”