Hierarchical Ensemble Kalman Methods with Sparsity-Promoting Generalized Gamma Hyperpriors

Kim, Hwanwoo; Sanz-Alonso, Daniel; Strang, Alexander

doi:10.48550/arxiv.2205.09322

Cited by 4 publications

(5 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, future studies will address the possibility of exploiting recent computational frameworks incorporating effective sparsity-promoting regularization techniques (e.g. relying on efficient hierarchical ensemble Kalman methods [64] or variational Bayesian inference [65]) to improve the capabilities of the MR-BCS-CSI.…”

Section: Discussionmentioning

confidence: 99%

Contrast source inversion of sparse targets through multi-resolution Bayesian compressive sensing

Salucci,

Poli,

Zardi

et al. 2024

Inverse Problems

View full text Add to dashboard Cite

The retrieval of non-Born scatterers is addressed within the contrast source inversion (CSI) framework by means of a novel multi-step inverse scattering (IS) method that jointly exploits prior information on the class of targets under investigation and progressively-acquired knowledge on the domain under investigation. The multi-resolution (MR) representation of the unknown contrast sources is iteratively retrieved by applying a Bayesian compressive sensing (BCS) sparsity-promoting approach based on a constrained relevance vector machine (C-RVM) solver. Representative examples of inversions from synthetic and experimental data are reported to give some indications on the reliability and the robustness of the proposed MR-BCS-CSI method. Comparisons with recent and competitive state-of-the-art (SoA) alternatives are reported, as well.

show abstract

Section: Discussionmentioning

confidence: 99%

Contrast source inversion of sparse targets through multi-resolution Bayesian compressive sensing

Salucci,

Poli,

Zardi

et al. 2024

Inverse Problems

View full text Add to dashboard Cite

show abstract

“…For all these algorithms based on a Gaussian ansatz, the accuracy depends on some measure of being close to Gaussian. Regarding the use of Gaussian approximations in Kalman inversion we highlight, in addition to the approximate Bayesian methods already cited, the use of ensemble Kalman methods for optimization: see [64,24,76,63,135]. Kalman filtering has also been used in combination with variational inference [78].…”

Section: Gaussian Approximationsmentioning

confidence: 99%

Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance

Chen¹,

Huang²,

Huang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Sampling a probability distribution with an unknown normalization constant is a fundamental problem in computational science and engineering. This task may be cast as an optimization problem over all probability measures, and an initial distribution can be evolved to the desired minimizer (the target distribution) dynamically via gradient flows. Mean-field models, whose law is governed by the gradient flow in the space of probability measures, may also be identified; particle approximations of these mean-field models form the basis of algorithms. The gradient flow approach is also the basis of algorithms for variational inference, in which the optimization is performed over a parameterized family of probability distributions such as Gaussians, and the underlying gradient flow is restricted to the parameterized family.By choosing different energy functionals and metrics for the gradient flow, different algorithms with different convergence properties arise. In this paper, we concentrate on the Kullback-Leibler divergence as the energy functional after showing that, up to scaling, it has the unique property (among all f -divergences) that the gradient flows resulting from this choice of energy do not depend on the normalization constant of the target distribution. For the metrics, we focus on variants of the Fisher-Rao, Wasserstein, and Stein metrics; we introduce the affine invariance property for gradient flows, and their corresponding mean-field models, determine whether a given metric leads to affine invariance, and modify it to make it affine invariant if it does not.We study the resulting gradient flows in both the space of all probability density functions and in the subset of all Gaussian densities. The flow in the Gaussian space may be understood as a Gaussian approximation of the flow in the density space. We demonstrate that, under mild assumptions, the Gaussian approximation based on the metric and through moment closure coincide; the moment closure approach is more convenient for calculations. We establish connections between these approximate gradient flows, discuss their relation to natural gradient methods in parametric variational inference, and study their long-time convergence properties showing, for some classes of problems and metrics, the advantages of affine invariance. Furthermore, numerical experiments are included which demonstrate that affine invariant gradient flows have desirable convergence properties for a wide range of highly anisotropic target distributions.

show abstract

“…Ensemble-based optimization schemes often rely on statistical linearization to avoid the computation of derivatives. Underpinning this idea [95,19,52] is the argument that if G(u) = Au were linear, then C up = CA ⊤ , leading to the approximation in the general nonlinear case…”

Section: Ensemble Algorithms For Sequential Optimizationmentioning

confidence: 99%

Non-Asymptotic Analysis of Ensemble Kalman Updates: Effective Dimension and Localization

Ghattas¹,

Sanz-Alonso²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Many modern algorithms for inverse problems and data assimilation rely on ensemble Kalman updates to blend prior predictions with observed data. Ensemble Kalman methods often perform well with a small ensemble size, which is essential in applications where generating each particle is costly. This paper develops a non-asymptotic analysis of ensemble Kalman updates that rigorously explains why a small ensemble size suffices if the prior covariance has moderate effective dimension due to fast spectrum decay or approximate sparsity. We present our theory in a unified framework, comparing several implementations of ensemble Kalman updates that use perturbed observations, square root filtering, and localization. As part of our analysis, we develop new dimension-free covariance estimation bounds for approximately sparse matrices that may be of independent interest.

show abstract

Hierarchical Ensemble Kalman Methods with Sparsity-Promoting Generalized Gamma Hyperpriors

Cited by 4 publications

References 42 publications

Contrast source inversion of sparse targets through multi-resolution Bayesian compressive sensing

Contrast source inversion of sparse targets through multi-resolution Bayesian compressive sensing

Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance

Non-Asymptotic Analysis of Ensemble Kalman Updates: Effective Dimension and Localization

Contact Info

Product

Resources

About