Convergence acceleration of ensemble Kalman inversion in nonlinear settings

Abstract

Many data-science problems can be formulated as an inverse problem, where the parameters are estimated by minimizing a proper loss function. When complicated black-box models are involved, derivative-free optimization tools are often needed. The ensemble Kalman filter (EnKF) is a particle-based derivative-free Bayesian algorithm originally designed for data assimilation. Recently, it has been applied to inverse problems for computational efficiency. The resulting algorithm, known as ensemble Kalman inversion (EKI), involves running an ensemble of particles with EnKF update rules so they can converge to a minimizer. In this article, we investigate EKI convergence in general nonlinear settings. To improve convergence speed and stability, we consider applying EKI with non-constant step-sizes and covariance inflation. We prove that EKI can hit critical points with finite steps in non-convex settings. We further prove that EKI converges to the global minimizer polynomially fast if the loss function is strongly convex. We verify the analysis presented with numerical experiments on two inverse problems.

Ensemble Kalman Inversion for nonlinear problems: weights, consistency, and variance bounds

“…Since EnKI was proposed in [15], the mystery of if and how it works for the nonlinear case has attracted a lot of attention. The surrounding work, such as the wellposedness of the coupled SDE [4], the wellposedness of the PDE [8,9], the mean-field limit of SDE to the PDE, and the convergence rate [8,13], and convergence as an optimization method [6,7], have all been studied in depth, and the use of similar idea leads to development of new algorithms [18,9]. The investigation into the core sampling problem with nonlinear forward map, however, is thin.…”

“…solution to the PDE, but the method nevertheless presents the flow to the PDE, so the method does not give a consistent sampling of the target distribution. We also note that often in time, people view EnKI as an optimization algorithm instead of a sampling algorithm, and some relaxation terms have been added for convergence to the minimizer [6,7].…”

“…We note that some sampling algorithms cannot provide i.i.d. {w n , u n } pairs, making the inequality (7) not exactly true. It still provides a good estimate in the mean-field regime when N 1.…”

Continuous time limit of the stochastic ensemble Kalman inversion: Strong convergence analysis

“…In [5] the authors extend the results from [60] by showing well-posedness of the stochastic formulation and deriving first convergence results for linear forward models. The EKI for nonlinear forward models has been studied in [11] in discrete time with nonconstant step size. In [7] the dynamical system resulting from the continuous time limit of the EKI has been described and analysed by a spectral decomposition.…”

“…This assumption can be implemented by modifying the underlying forward map with a smooth shift to 0 close to the boundary of u ∈ (−M, M ). The EKI has been analysed under this assumption for example in [11,10].…”