A data-scalable randomized misfit approach for solving large-scale PDE-constrained inverse problems

Abstract

Abstract. A randomized misfit approach is presented for the efficient solution of large-scale PDE-constrained inverse problems with high-dimensional data. The purpose of this paper is to offer a theory-based framework for random projections in this inverse problem setting. The stochastic approximation to the misfit is analyzed using random projection theory. By expanding beyond mean estimator convergence, a practical characterization of randomized misfit convergence can be achieved. The theoretical results developed hold with any valid random projection in the literature. The class of feasible distributions is broad yet simple to characterize compared to previous stochastic misfit methods. This class includes very sparse random projections which provide additional computational benefit. A different proof for a variant of the Johnson-Lindenstrauss lemma is also provided. This leads to a different intuition for the O(ε −2 ) factor in bounds for Johnson-Lindenstrauss results. The main contribution of this paper is a theoretical result showing the method guarantees a valid solution for small reduced misfit dimensions. The interplay between Johnson-Lindenstrauss theory and Morozov's discrepancy principle is shown to be essential to the result. The computational cost savings for large-scale PDE-constrained problems with highdimensional data is discussed. Numerical verification of the developed theory is presented for model problems of estimating a distributed parameter in an elliptic partial differential equation. Results with different random projections are presented to demonstrate the viability and accuracy of the proposed approach.

Random Forward Models and Log-Likelihoods in Bayesian Inverse Problems

“…This section considers a particular Monte Carlo approximation Φ N of a quadratic potential Φ, proposed by Nemirovski et al (2008); Shapiro et al (2009), and further applied and analysed in the context of BIPs by Le et al (2017). This approximation is particularly useful when the data y ∈ R J has very high dimension, so that one does not wish to interrogate every component of the data vector y, or evaluate every component of the model prediction G(u) and compare it with the corresponding component of y.…”

“…This task involves the solution of a large-scale optimisation problem involving Φ in the objective function, which is typically done using inexact Newton methods. It is shown by Le et al (2017) that the required number of evaluations of the forward model G and its adjoint is drastically reduced when using the randomised misfit Φ N as opposed to using the true misfit Φ, approximately by a factor of J N . The aim of this section is to show that the use of the randomised misfit Φ N does not only lead to the MAP estimate being well-approximated, but in fact the whole Bayesian posterior distribution.…”

“…For example, an expensive forward model such as the solution of a PDE may replaced by a kriging/Gaussian process (GP) model (Stuart and Teckentrup, 2017). In the realm of "big data" a residual vector of prohibitively high dimension may be randomly subsampled or orthogonally projected onto a randomly-chosen lowdimensional subspace (Le et al, 2017;Nemirovski et al, 2008). In the field of probabilistic numerical methods (Hennig et al, 2015), a deterministic dynamical system may be solved stochastically, with the stochasticity representing epistemic uncertainty about the behaviour of the system below the temporal or spatial grid scale (Conrad et al, 2016;Lie et al, 2017).…”

Sampling Methods for Bayesian Inference Involving Convergent Noisy Approximations of Forward Maps

“…In particular, one can think of random misfit models, in which the likelihood is computed only on a random subset of the data (see e.g. [27,30]), or of multiscale models, where fast-scale effects can be modelled as random and one infers an effective slow-scale map from multiscale data [18]. Another possible application is given by probabilistic numerical methods (see e.g.…”

Error bounds for some approximate posterior measures in Bayesian inference

“…[4,5] and the references therein. In other cases, randomisation is used to reduce the computational cost of an existing method, for example in Markov chain Monte Carlo [6,7,8], sampling the posterior [9], or dimension reduction for solving inverse problems [10]. The results that we present below are motivated by the use of randomisation in problems where computation with the exact likelihood or forward model is not computationally efficient or feasible, for example the use of Gaussian process approximations of the negative log-likelihood and forward model [11].…”