Fast computation of uncertainty quantification measures in the geostatistical approach to solve inverse problems

Saibaba, Arvind K.; Kitanidis, Peter K.

doi:10.1016/j.advwatres.2015.04.012

Cited by 39 publications

(62 citation statements)

References 37 publications

(109 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For several inverse problems the eigenvalues of the eigenproblem (16) decay rapidly so that the low-rank approximation can be truncated for small k resulting in an efficient representation of the posterior covariance matrix. We will discuss this application in an upcoming paper [22].…”

Section: Discussionmentioning

confidence: 99%

Randomized algorithms for generalized Hermitian eigenvalue problems with application to computing Karhunen–Loève expansion

Saibaba

Lee

Kitanidis

2015

Numerical Linear Algebra App

Self Cite

View full text Add to dashboard Cite

SUMMARYWe describe randomized algorithms for computing the dominant eigenmodes of the Generalized Hermitian Eigenvalue Problem (GHEP) Ax = λBx, with A Hermitian and B Hermitian and positive definite. The algorithms we describe only require forming operations Ax, Bx and B −1 x and avoid forming square-roots of B (or operations of the form, B 1/2 x or B −1/2 x). We provide a convergence analysis and a posteriori error bounds that build upon the work of [13,16,18] (which have been derived for the case B = I). Additionally, we derive some new results that provide insight into the accuracy of the eigenvalue calculations. The error analysis shows that the randomized algorithm is most accurate when the generalized singular values of B −1 A decay rapidly. A randomized algorithm for the Generalized Singular Value Decomposition (GSVD) is also provided. Finally, we demonstrate the performance of our algorithm on computing the KarhunenLoève expansion, which is a computationally intensive GHEP problem with rapidly decaying eigenvalues.

show abstract

Section: Discussionmentioning

confidence: 99%

Randomized algorithms for generalized Hermitian eigenvalue problems with application to computing Karhunen–Loève expansion

Saibaba

Lee

Kitanidis

2015

Numerical Linear Algebra App

Self Cite

View full text Add to dashboard Cite

show abstract

“…yields the so-called prior-preconditioned Hessian transformation H := L − A AL −1 [12,16,47,57]. For highly ill-posed inverse problems such as those considered here, A either has a rapidly decaying spectrum or is rank deficient.…”

Section: Approximating the Target Distributionmentioning

confidence: 99%

Low-Rank Independence Samplers in Hierarchical Bayesian Inverse Problems

Brown¹,

Saibaba²,

Vallélian³

2018

SIAM/ASA J. Uncertainty Quantification

View full text Add to dashboard Cite

In Bayesian inverse problems, the posterior distribution is used to quantify uncertainty about the reconstructed solution. In practice, Markov chain Monte Carlo algorithms often are used to draw samples from the posterior distribution. However, implementations of such algorithms can be computationally expensive. We present a computationally efficient scheme for sampling high-dimensional Gaussian distributions in ill-posed Bayesian linear inverse problems. Our approach uses Metropolis-Hastings independence sampling with a proposal distribution based on a low-rank approximation of the prior-preconditioned Hessian. We show the dependence of the acceptance rate on the number of eigenvalues retained and discuss conditions under which the acceptance rate is high. We demonstrate our proposed sampler by using it with Metropolis-Hastings-within-Gibbs sampling in numerical experiments in image deblurring, computerized tomography, and NMR relaxometry.

show abstract

“…A matrix-free approach for solving the multi-parametric Gaussian maximum likelihood problem was developed in [4]. To further improve on these issues, other methods that have been recently developed include the nearest-neighbor Gaussian process models [17], low-rank update [66], multiresolution Gaussian process models [57], equivalent kriging [42], multi-level restricted Gaussian maximum likelihood estimators [14], and hierarchical low-rank approximations [35]. Bayesian approaches to identify unknown or uncertain parameters could be also applied [62,61,55,50,55,58].In this paper, we propose using the so-called hierarchical (H-) matrices for approximating dense matrices with numerical complexity and storage O(k α n log α n), where n is the number of measurements; k n is the rank of the hierarchical matrix, which defines the quality of the approximation; and α = 1 or 2.…”

mentioning

confidence: 99%

Likelihood approximation with hierarchical matrices for large spatial datasets

Litvinenko

Sun

Genton

et al. 2019

Computational Statistics & Data Analysis

View full text Add to dashboard Cite

We consider measurements to estimate the unknown parameters (variance, smoothness, and covariance length) of a covariance function by maximizing the joint Gaussian log-likelihood function. To overcome cubic complexity in the linear algebra, we approximate the discretized covariance function in the hierarchical (H-) matrix format. The H-matrix format has a log-linear computational cost and storage O(kn log n), where the rank k is a small integer, and n is the number of locations. The H-matrix technique allows us to work with general covariance matrices efficiently, since H-matrices can approximate inhomogeneous covariance functions, with a fairly general mesh that is not necessarily axes-parallel, and neither the covariance matrix itself nor its inverse has to be sparse. We research how the H-matrix approximation error influences on the estimated parameters. We demonstrate our method with Monte Carlo simulations with known true values of parameters and an application to soil moisture data with unknown parameters. The C, C++ codes and data are freely available. becomes challenging, due to the computation of the inverse and log-determinant of the n-by-n covariance matrix C(θ). Indeed, this requires O(n 2 ) memory and O(n 3 ) computational steps. Hence, scalable methods that can process larger sample sizes are needed.Stationary covariance functions, discretized on a rectangular grid, have block Toeplitz structure. This structure can be further extended to a block circulant form and resolved with the Fast Fourier Transform (FFT) [79,16,25,74,18]. The computing cost, in this case, is O(n log n). However, this approach either does not work for data measured at irregularly spaced locations or requires expensive, non-trivial modifications.During the past two decades, a large amount of research has been devoted to tackling the aforementioned computational challenge of developing scalable methods: for example, low-rank tensor methods [52,56], covariance tapering [21,38,68], likelihood approximations in both the spatial [72,73] and spectral [19] domains, latent processes such as Gaussian predictive processes [6] and fixed-rank kriging [15], and Gaussian Markov random-field approximations [64,63,20]; see [77] for a review. A generalization of the Vecchia approach and a general Vecchia framework was introduced in [37, 78]. Each of these methods has its strengths and drawbacks. For instance, covariance tapering sometimes performs even worse than assuming independent blocks in the covariance [70]; low-rank approximations have their own limitations [71]; and Markov models depend on the observation locations, requiring irregular locations to be realigned on a much finer grid with missing values [76]. A matrix-free approach for solving the multi-parametric Gaussian maximum likelihood problem was developed in [4]. To further improve on these issues, other methods that have been recently developed include the nearest-neighbor Gaussian process models [17], low-rank update [66], multiresolution Gaussian process models [57], equivalent ...

show abstract

Fast computation of uncertainty quantification measures in the geostatistical approach to solve inverse problems

Cited by 39 publications

References 37 publications

Randomized algorithms for generalized Hermitian eigenvalue problems with application to computing Karhunen–Loève expansion

Randomized algorithms for generalized Hermitian eigenvalue problems with application to computing Karhunen–Loève expansion

Low-Rank Independence Samplers in Hierarchical Bayesian Inverse Problems

Likelihood approximation with hierarchical matrices for large spatial datasets

Contact Info

Product

Resources

About