Sampling-free Bayesian inversion with adaptive hierarchical tensor representations

Eigel, Martin; Marschall, Manuel; Schneider, Reinhold

doi:10.1088/1361-6420/aaa998

Cited by 15 publications

(21 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It should be noted that a functional adaptive evluation of the forward map allows for the derivation of an explicit adaptive Bayesian inversion with functional tensor representations as in [17]. The results of the present work lay the ground for a similar approach with a Gaussian prior assumption.…”

Section: Introductionmentioning

confidence: 62%

Adaptive stochastic Galerkin FEM for lognormal coefficients in hierarchical tensor representations

et al. 2020

Self Cite

View full text Add to dashboard Cite

Stochastic Galerkin methods for non-affine coefficient representations are known to cause major difficulties from theoretical and numerical points of view. In this work, an adaptive Galerkin FE method for linear parametric PDEs with lognormal coefficients discretized in Hermite chaos polynomials is derived. It employs problem-adapted function spaces to ensure solvability of the variational formulation. The inherently high computational complexity of the parametric operator is made tractable by using hierarchical tensor representations. For this, a new tensor train format of the lognormal coefficient is derived and verified numerically. The central novelty is the derivation of a reliable residual-based a posteriori error estimator. This can be regarded as a unique feature of stochastic Galerkin methods. It allows for an adaptive algorithm to steer the refinements of the physical mesh and the anisotropic Wiener chaos polynomial degrees. For the evaluation of the error estimator to become feasible, a numerically efficient tensor format discretization is developed. Benchmark examples with unbounded lognormal coefficient fields illustrate the performance of the proposed Galerkin discretization and the fully adaptive algorithm.

show abstract

Section: Introductionmentioning

confidence: 62%

Adaptive stochastic Galerkin FEM for lognormal coefficients in hierarchical tensor representations

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is motivated by our previous work on adaptive low-rank approximations of solutions of parametric random PDEs with Adaptive Stochastic Galerkin FEM (ASGFEM, see e.g. [20,17]) and in particular the sampling-free Bayesian inversion presented in [18] where the setting of uniform random variables was examined. A generalization to the important case of Gaussian random variables turns out to be non-trivial from a computational point of view due to the difficulties of representing highly concentrated densities in a compressing tensor format which is required in order to cope with the high dimensionality of the problem.…”

Section: Overviewmentioning

confidence: 99%

“…We recall the general formalism and highlight the notation with the setup of Section 2 in mind. We closely follow the presentation in [18] and refer to [54,10,33] for a comprehensive overview.…”

Section: Applicationsmentioning

confidence: 99%

Low-rank tensor reconstruction of concentrated densities with application to Bayesian inversion

Eigel¹,

Gruhlke²,

Marschall³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Transport maps have become a popular mechanic to express complicated probability densities using sample propagation through an optimized pushforward. Beside their broad applicability and well-known success, transport maps suffer from several drawbacks such as numerical inaccuracies induced by the optimization process and the fact that sampling schemes have to be employed when quantities of interest, e.g. moments are to compute. This paper presents a novel method for the accurate functional approximation of probability density functions (PDF) that copes with those issues. By interpreting the pull-back result of a target PDF through an inexact transport map as a perturbed reference density, a subsequent functional representation in a more accessible format allows for efficient and more accurate computation of the desired quantities. We introduce a layer-based approximation of the perturbed reference density in an appropriate coordinate system to split the high-dimensional representation problem into a set of independent approximations for which separately chosen orthonormal basis functions are available. This effectively motivates the notion of h-and p-refinement (i.e. "mesh size" and polynomial degree) for the approximation of high-dimensional PDFs. To circumvent the curse of dimensionality and enable sampling-free access to certain quantities of interest, a low-rank reconstruction in the tensor train format is employed via the Variational Monte Carlo method. An a priori convergence analysis of the developed approach is derived in terms of Hellinger distance and the Kullback-Leibler divergence. Applications comprising Bayesian inverse problems and several degrees of concentrated densities illuminate the (superior) convergence in comparison to Monte Carlo and Markov-Chain Monte Carlo methods.

show abstract

“…Approximation of the multivariate target distribution can be recommended for the following two cases: First, the quantity of interest may be very poorly representable in the TT format, and hence direct tensor product integration of the QoI, as suggested in [11], is not possible. The most remarkable example is the indicator function, which occurs in the computation of the probability of an event.…”

Section: Introductionmentioning

confidence: 99%

Approximation and sampling of multivariate probability distributions in the tensor train decomposition

et al. 2019

View full text Add to dashboard Cite

General multivariate distributions are notoriously expensive to sample from, particularly the high-dimensional posterior distributions in PDE-constrained inverse problems. This paper develops a sampler for arbitrary continuous multivariate distributions that is based on low-rank surrogates in the tensor-train format, a methodology that has been exploited for many years for scalable, high-dimensional density function approximation in quantum physics and chemistry. We build upon recent developments of the cross approximation algorithms in linear algebra to construct a tensor-train approximation to the target probability density function using a small number of function evaluations. For sufficiently smooth distributions the storage required for accurate tensor-train approximations is moderate, scaling linearly with dimension. In turn, the structure of the tensor-train surrogate allows sampling by an efficient conditional distribution method since marginal distributions are computable with linear complexity in dimension. Expected values of non-smooth quantities of interest, with respect to the surrogate distribution, can be estimated using transformed independent uniformly-random seeds that provide Monte Carlo quadrature, or transformed points from a quasi-Monte Carlo lattice to give more efficient quasi-Monte Carlo quadrature. Unbiased estimates may be calculated by correcting the transformed random seeds using a Metropolis-Hastings accept/reject step, while the quasi-Monte Carlo quadrature may be corrected either by a control-variate strategy, or by importance weighting. We show that the error in the tensor-train approximation propagates linearly into the Metropolis-Hastings rejection rate and the integrated autocorrelation time of the resulting Markov chain; thus the integrated autocorrelation time may be made arbitrarily close to 1, implying that, asymptotic in sample size, the cost per effectively independent sample is one target density evaluation plus the cheap tensor-train surrogate proposal that has linear cost with dimension. These methods are demonstrated in three computed examples: fitting failure time of shock absorbers; a PDE-constrained inverse diffusion problem; and sampling from the Rosenbrock distribution. The delayed rejection adaptive Metropolis (DRAM) algorithm is used as a benchmark. In all computed examples, the importance-weight corrected quasi-Monte Carlo quadrature performs best, and is more efficient than DRAM by orders of magnitude across a wide range of approximation accuracies and sample sizes. Indeed, all the methods developed here significantly outperform DRAM in all computed examples.

show abstract

Sampling-free Bayesian inversion with adaptive hierarchical tensor representations

Cited by 15 publications

References 54 publications

Adaptive stochastic Galerkin FEM for lognormal coefficients in hierarchical tensor representations

Adaptive stochastic Galerkin FEM for lognormal coefficients in hierarchical tensor representations

Low-rank tensor reconstruction of concentrated densities with application to Bayesian inversion

Approximation and sampling of multivariate probability distributions in the tensor train decomposition

Contact Info

Product

Resources

About