Ensemble Kalman Inversion for General Likelihoods

Abstract

Ensemble Kalman inversion represents a powerful technique for inference in statistical models with likelihoods of the form y | x ∼ N(y | H(x), R) where the forward operator H and covariance R are known. In this article, we generalise ensemble Kalman inversion to models with general likelihoods, y | x ∼ p(y | x) where the likelihood can be sampled from, but its density not necessarily evaluated. We examine the ensemble Kalman performance for both optimisation and uncertainty quantification against fully adaptive approximate Bayesian computation techniques.

Component-wise iterative ensemble Kalman inversion for static Bayesian models with unknown measurement error covariance

“…Another avenue of future work is to investigate how our approach can be incorporated into the IEKI method of Duffield and Singh (2021) for general likelihoods. In this case, it may be possible to update some of the model parameters with IEKI and some with MCMC, depending on the form of the likelihood function.…”

“…In particular, one extension here is developing a likelihood tempering SMC algorithm with our CW-IEKI method as the forward kernel. It may also be possible to extend their SMC algorithm to general likelihood models, such that a subset of the parameters are updated using the method of Duffield and Singh (2021), and the rest are updated using an MCMC forward kernel.…”

“…to obtain samples from p(θ | y), early stopping of the algorithm is essential (Iglesias et al, 2013). If G(•) is a linear function and the prior is Gaussian, EKI provides exact samples from the posterior in a single iteration (J = 1) (Iglesias, 2014;Duffield and Singh, 2021), and the ensemble converges to the maximum a posteriori (MAP) estimate as J → ∞ (Iglesias et al, 2013;Duffield and Singh, 2021).…”

“…The ensemble Kalman sampler (Garbuno-Inigo et al, 2020;Ding and Li, 2021) is a variation of EKI, which perturbs the ensemble instead of the observations as in regular EKI. Duffield and Singh (2021) extend EKI for non-Gaussian likelihoods through a Gaussian approximation of the likelihood, and Wu et al (2022) use EKI as the forward kernel in data annealing SMC. The latter method is exact, and requires much fewer evaluations of G(•) than SMC with a Metropolis-Hastings forward kernel.…”