Consensus-based Optimization and Ensemble Kalman Inversion for Global Optimization Problems with Constraints

Carrillo, José Antonio Ortega; Totzeck, Claudia; Vaes, Urbain

doi:10.48550/arxiv.2111.02970

Cited by 4 publications

(4 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the first two tests, we compare the proposed Kalman inversion methods (EKI, UKI, EAKI, ETKI applied to Eqs. ( 14), ( 15) and ( 18) with other recently proposed Gaussian approximation algorithms, including the ensemble Kalman sampler (EKS) [75,77] + , and the consensus-based sampler (CBS) [103,51,104,105,52] * . We also compare with variants of iterative Kalman filter methods, which seek to deform the prior into the posterior in one time unit (transport/coupling) using a finite number of intermediate steps (see Appendix B) based on (12).…”

Section: Overview Of Test Problemsmentioning

confidence: 99%

Efficient Derivative-free Bayesian Inference for Large-Scale Inverse Problems

Huang¹,

Huang²,

Reich³

et al. 2022

Preprint

View full text Add to dashboard Cite

We consider Bayesian inference for large scale inverse problems, where computational challenges arise from the need for repeated evaluations of an expensive forward model. This renders most Markov chain Monte Carlo approaches infeasible, since they typically require O(10 4 ) model runs, or more. Moreover, the forward model is often given as a black box or is impractical to differentiate. Therefore derivative-free algorithms are highly desirable. We propose a framework, which is built on Kalman methodology, to efficiently perform Bayesian inference in such inverse problems. The basic method is based on an approximation of the filtering distribution of a novel mean-field dynamical system into which the inverse problem is embedded as an observation operator. Theoretical properties of the mean-field model are established for linear inverse problems, demonstrating that the desired Bayesian posterior is given by the steady state of the law of the filtering distribution of the mean-field dynamical system, and proving exponential convergence to it. This suggests that, for nonlinear problems which are close to Gaussian, sequentially computing this law provides the basis for efficient iterative methods to approximate the Bayesian posterior. Ensemble methods are applied to obtain interacting particle system approximations of the filtering distribution of the mean-field model; and practical strategies to further reduce the computational and memory cost of the methodology are presented, including low-rank approximation and a bi-fidelity approach. The effectiveness of the framework is demonstrated in several numerical experiments, including proofof-concept linear/nonlinear examples and two large-scale applications: learning of

show abstract

Section: Overview Of Test Problemsmentioning

confidence: 99%

Efficient Derivative-free Bayesian Inference for Large-Scale Inverse Problems

Huang¹,

Huang²,

Reich³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In the first two tests, we compare the proposed Kalman inversion methods (EKI, UKI, EAKI, ETKI applied to equations ( 14), ( 15) and ( 18) with other recently proposed Gaussian approximation algorithms, including the ensemble Kalman sampler (EKS) [75,77], 9 and the consensus-based sampler (CBS) [52,53,[103][104][105]. 10 We also compare with variants of iterative Kalman filter methods, which seek to deform the prior into the posterior in one time unit (transport/coupling) using a finite number of intermediate steps (see appendix B) based on (12).…”

Section: Overview Of Test Problemsmentioning

confidence: 99%

Efficient derivative-free Bayesian inference for large-scale inverse problems

Huang¹,

Huang²,

Reich³

et al. 2022

Inverse Problems

View full text Add to dashboard Cite

We consider Bayesian inference for large-scale inverse problems, where computational challenges arise from the need for repeated evaluations of an expensive forward model. This renders most Markov chain Monte Carlo approaches infeasible, since they typically require O(104) model runs, or more. Moreover, the forward model is often given as a black box or is impractical to differentiate. Therefore derivative-free algorithms are highly desirable. We propose a framework, which is built on Kalman methodology, to efficiently perform Bayesian inference in such inverse problems. The basic method is based on an approximation of the filtering distribution of a novel mean-field dynamical system, into which the inverse problem is embedded as an observation operator. Theoretical properties are established for linear inverse problems, demonstrating that the desired Bayesian posterior is given by the steady-state of the law of the filtering distribution of the mean-field dynamical system, and proving exponential convergence to it. This suggests that, for nonlinear problems which are close to Gaussian, sequentially computing this law provides the basis for efficient iterative methods to approximate the Bayesian posterior. Ensemble methods are applied to obtain interacting particle system approximations of the filtering distribution of the mean-field model; and practical strategies to further reduce the computational and memory cost of the methodology are presented, including low-rank approximation and a bi-fidelity approach. The effectiveness of the framework is demonstrated in several numerical experiments, including proof-of-concept linear/nonlinear examples and two large-scale applications: learning of permeability parameters in subsurface flow; and learning subgrid-scale parameters in a global climate model. Moreover, the stochastic ensemble Kalman filter and various ensemble square-root Kalman filters are all employed and are compared numerically. The results demonstrate that the proposed method, based on exponential convergence to the filtering distribution of a mean-field dynamical system, is competitive with pre-existing Kalman-based methods for inverse problems.

show abstract

“…Here, we do not require any additional smoothing or any special treatment of the non-differentiability, since CBO does not rely on explicit gradient information. At the same time as finalizing this manuscript, we learned that the case of smooth penalization in the CBO context has been discussed in [12]. In their approach however the penalty parameter needs to tend to infinity and contrary to our work no explicit update rule of the parameter is given.…”

Section: Introductionmentioning

confidence: 96%

Constrained consensus-based optimization

Borghi¹,

Herty²,

Pareschi³

2021

Preprint

View full text Add to dashboard Cite

In this work we are interested in the construction of numerical methods for high dimensional constrained nonlinear optimization problems by particle-based gradient-free techniques. A consensus-based optimization (CBO) approach combined with suitable penalization techniques is introduced for this purpose. The method relies on a reformulation of the constrained minimization problem in an unconstrained problem for a penalty function and extends to the constrained settings the class of CBO methods. Exact penalization is employed and, since the optimal penalty parameter is unknown, an iterative strategy is proposed that successively updates the parameter based on the constrained violation. Using a mean-field description of the the many particle limit of the arising CBO dynamics, we are able to show convergence of the proposed method to the minimum for general nonlinear constrained problems. Properties of the new algorithm are analyzed. Several numerical examples, also in high dimensions, illustrate the theoretical findings and the good performance of the new numerical method.

show abstract

Consensus-based Optimization and Ensemble Kalman Inversion for Global Optimization Problems with Constraints

Cited by 4 publications

References 44 publications

Efficient Derivative-free Bayesian Inference for Large-Scale Inverse Problems

Efficient Derivative-free Bayesian Inference for Large-Scale Inverse Problems

Efficient derivative-free Bayesian inference for large-scale inverse problems

Constrained consensus-based optimization

Contact Info

Product

Resources

About