Ergodicity of the underdamped mean-field Langevin dynamics

Kazeykina, Anna; Ren, Zhenjie; Tan, Xiaolu; Yang, Junjian

doi:10.48550/arxiv.2007.14660

Cited by 6 publications

(10 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…( 8)), and the convergence rate of which is more difficult to establish. (Monmarché, 2017;Guillin et al, 2019;Kazeykina et al, 2020;Guillin et al, 2021), based on hypocoercivity (Villani, 2009) or coupling techniques (Eberle et al, 2019). In addition, Bou-Rabee and Schuh (2020); Bou-Rabee and Eberle (2021) studied the dis-crete time convergence of Hamiltonian Monte Carlo with interaction potential.…”

Section: Related Literaturementioning

confidence: 99%

“…Recent works have studied the convergence rate of the mean field Langevin dynamics, including its underdamped (kinetic) version. However, most existing analyses either require sufficiently strong regularization (Hu et al, 2019;Jabir et al, 2019), or build upon involved mathematical tools (Kazeykina et al, 2020;Guillin et al, 2021). Our goal is to provide a simpler convergence proof that covers general and more practical machine learning settings, with a focus on neural network optimization in the mean field regime.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Convex Analysis of the Mean Field Langevin Dynamics

Nitanda¹,

Wu²,

Suzuki³

2022

Preprint

View full text Add to dashboard Cite

As an example of the nonlinear Fokker-Planck equation, the mean field Langevin dynamics recently attracts attention due to its connection to (noisy) gradient descent on infinitely wide neural networks in the mean field regime, and hence the convergence property of the dynamics is of great theoretical interest. In this work, we give a simple and self-contained convergence rate analysis of the mean field Langevin dynamics with respect to the (regularized) objective function in both continuous and discrete time settings. The key ingredient of our proof is a proximal Gibbs distribution pq associated with the dynamics, which, in combination of techniques in Vempala and Wibisono (2019), allows us to develop a convergence theory parallel to classical results in convex optimization. Furthermore, we reveal that pq connects to the duality gap in the empirical risk minimization setting, which enables efficient empirical evaluation of the algorithm convergence.

show abstract

Section: Related Literaturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Convex Analysis of the Mean Field Langevin Dynamics

Nitanda¹,

Wu²,

Suzuki³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The particle interacts with its environment through the non-local term K * ρ whilst being confined by an external potential g, experiencing a frictional force f and a diffusive force Wt. We have just presented a statistical physics interpretation of (5.3), however, more recently there has been a resurgence of interest in such equations among those in the machine learning community who want to rigorously prove trainability of neural networks, see [KRTY20] and references therein.…”

Section: Vlasov-fokker-planck Equation (Vfpe)mentioning

confidence: 99%

Operator-splitting schemes for degenerate, non-local, conservative-dissipative systems

Adams¹,

Duong²,

Reis³

2021

Preprint

View full text Add to dashboard Cite

The theory of Wasserstein gradient flows in the space of probability measures provides a powerful framework to study dissipative partial differential equations (PDE). It can be used to prove well-posedness, regularity, stability and quantitative convergence to the equilibrium. However, many PDE are not gradient flows, and hence the theory is not immediately applicable.In this work we develop a straightforward entropy regularised splitting scheme for degenerate non-local non-gradient systems. The approach is composed of two main stages: first we split the dynamics into the conservative and dissipative forces, secondly we perturb the problem so that the diffusion is no longer singular and perform a weighted Wasserstein "JKO type" descent step. Entropic regularisation of optimal transport problems opens the way for efficient numerical methods for solving these gradient flows. We illustrate the generality of our work by providing a number of examples, including the Regularized Vlasov-Poisson-Fokker-Planck equation, to which our results applicable.

show abstract

“…Algorithms using Langevin dynamics have a better long-time behaviour compared to the overdamped Langevin dynamics [15,16], which forms a degenerated special case of the Langevin dynamics, where the limit for γ to infinity is taken [41,Section 6.5.1]. Therefore, nonlinear Langevin dynamics became recently popular for training networks as the Generative Adversarial Network (GAN) [33].…”

Section: Introductionmentioning

confidence: 99%

Global contractivity for Langevin dynamics with distribution-dependent forces and uniform in time propagation of chaos

Schuh¹

2022

Preprint

View full text Add to dashboard Cite

We study the long-time behaviour of both the classical second-order Langevin dynamics and the nonlinear second-order Langevin dynamics of McKean-Vlasov type. By a coupling approach, we establish global contraction in an L 1 Wasserstein distance with an explicit dimension-free rate for pairwise weak interactions. For external forces corresponding to a κ-strongly convex potential, a contraction rate of order O( √ κ) is obtained in certain cases. But the contraction result is not restricted to these forces. It rather includes multi-well potentials and non-gradient-type external forces as well as non-gradienttype repulsive and attractive interaction forces. The proof is based on a novel distance function which combines two contraction results for large and small distances and uses a coupling approach adjusted to the distance. By applying a componentwise adaptation of the coupling we provide uniform in time propagation of chaos bounds for the corresponding mean-field particle system.

show abstract

Ergodicity of the underdamped mean-field Langevin dynamics

Cited by 6 publications

References 30 publications

Convex Analysis of the Mean Field Langevin Dynamics

Convex Analysis of the Mean Field Langevin Dynamics

Operator-splitting schemes for degenerate, non-local, conservative-dissipative systems

Global contractivity for Langevin dynamics with distribution-dependent forces and uniform in time propagation of chaos

Contact Info

Product

Resources

About