On the mean-field limit for the consensus-based optimization

Huang, Hui; Qiu, Jinniao

doi:10.48550/arxiv.2105.12919

Cited by 6 publications

(10 citation statements)

References 19 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…and the initial data is attained pointwise, i.e., lim t→0 f (t) = f 0 . We refer to [38] for a detailed discussion and proof of the mean-field limit and to [10,23] for the well-posedness of (3.2). We recall, that (3.2) strongly depends on the function P through the term x α (f ), which is nonlinear and nonlocal.…”

Section: Mean-field Approximation and Main Resultsmentioning

confidence: 99%

“…Consensus-based optimization (CBO) methods have been introduced and studied recently as efficient computational tools for solving high dimensional nonlinear unconstrained minimization problems [4,10,11,13,15,29,32,38,47,49,50]. They belong to the family of individual-based models that are inspired by self-organized dynamics based on alignment [18,42,43,52].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Constrained consensus-based optimization

Borghi¹,

Herty²,

Pareschi³

2021

Preprint

View full text Add to dashboard Cite

In this work we are interested in the construction of numerical methods for high dimensional constrained nonlinear optimization problems by particle-based gradient-free techniques. A consensus-based optimization (CBO) approach combined with suitable penalization techniques is introduced for this purpose. The method relies on a reformulation of the constrained minimization problem in an unconstrained problem for a penalty function and extends to the constrained settings the class of CBO methods. Exact penalization is employed and, since the optimal penalty parameter is unknown, an iterative strategy is proposed that successively updates the parameter based on the constrained violation. Using a mean-field description of the the many particle limit of the arising CBO dynamics, we are able to show convergence of the proposed method to the minimum for general nonlinear constrained problems. Properties of the new algorithm are analyzed. Several numerical examples, also in high dimensions, illustrate the theoretical findings and the good performance of the new numerical method.

show abstract

Section: Mean-field Approximation and Main Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Constrained consensus-based optimization

Borghi¹,

Herty²,

Pareschi³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…with initial datum f 0 . The mean-field limit results [3,22,21,43,24,23] ensure that the particle system (1.2) is well approximated by the following self-consistent mean-field McKean process…”

Section: Introductionmentioning

confidence: 87%

On the Global Convergence of Particle Swarm Optimization Methods

Huang¹,

Qiu²,

Riedl³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper we provide a rigorous convergence analysis for the renowned Particle Swarm Optimization method using tools from stochastic calculus and the analysis of partial differential equations. Based on a time-continuous formulation of the particle dynamics as a system of stochastic differential equations, we establish the convergence to a global minimizer in two steps. First, we prove the consensus formation of the dynamics by analyzing the time-evolution of the variance of the particle distribution. Consecutively, we show that this consensus is close to a global minimizer by employing the asymptotic Laplace principle and a tractability condition on the energy landscape of the objective function. Our results allow for the usage of memory mechanisms, and hold for a rich class of objectives provided certain conditions of well-preparation of the hyperparameters and the initial datum are satisfied. To demonstrate the applicability of the method we propose an efficient and parallelizable implementation, which is tested in particular on a competitive and well-understood high-dimensional benchmark problem in machine learning.

show abstract

“…In the large particle limit an agent is not influenced by individual particles but only by the average behavior of all particles. As shown in [11], the empirical random agent measure ρ N converges in law to the deterministic agent density ρ ∈ C([0, T ], P(R d )), which weakly (see Definition 1) satisfies the non-linear Fokker-Planck equation…”

Section: Introductionmentioning

confidence: 99%

Convergence of Anisotropic Consensus-Based Optimization in Mean-Field Law

Fornasier,

Klock,

Riedl

2021

Preprint

View full text Add to dashboard Cite

In this paper we study anisotropic consensus-based optimization (CBO), a multi-agent metaheuristic derivative-free optimization method capable of globally minimizing nonconvex and nonsmooth functions in high dimensions. CBO is based on stochastic swarm intelligence, and inspired by consensus dynamics and opinion formation. Compared to other metaheuristic algorithms like particle swarm optimization, CBO is of a simpler nature and therefore more amenable to theoretical analysis. By adapting a recently established proof technique, we show that anisotropic CBO converges globally with a dimension-independent rate for a rich class of objective functions under minimal assumptions on the initialization of the method. Moreover, the proof technique reveals that CBO performs a convexification of the optimization problem as the number of agents goes to infinity, thus providing an insight into the internal CBO mechanisms responsible for the success of the method. To motivate anisotropic CBO from a practical perspective, we further test the method on a complicated high-dimensional benchmark problem, which is well understood in the machine learning literature.

show abstract

On the mean-field limit for the consensus-based optimization

Cited by 6 publications

References 19 publications

Constrained consensus-based optimization

Constrained consensus-based optimization

On the Global Convergence of Particle Swarm Optimization Methods

Convergence of Anisotropic Consensus-Based Optimization in Mean-Field Law

Contact Info

Product

Resources

About