Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: A stochastic control approach

Abstract

We study the convergence to equilibrium of an underdamped Langevin equation that is controlled by a linear feedback force. Specifically, we are interested in sampling the possibly multimodal invariant probability distribution of a Langevin system at small noise (or low temperature), for which the dynamics can easily get trapped inside metastable subsets of the phase space. We follow Chen et al. [J. Math. Phys. 56, 113302 (2015)] and consider a Langevin equation that is simulated at a high temperature, with the control playing the role of a friction that balances the additional noise so as to restore the original invariant measure at a lower temperature. We discuss different limits as the temperature ratio goes to infinity and prove convergence to a limit dynamics. It turns out that, depending on whether the lower (“target”) or the higher (“simulation”) temperature is fixed, the controlled dynamics converges either to the overdamped Langevin equation or to a deterministic gradient flow. This implies that (a) the ergodic limit and the large temperature separation limit do not commute in general and that (b) it is not possible to accelerate the speed of convergence to the ergodic limit by making the temperature separation larger and larger. We discuss the implications of these observations from the perspective of stochastic optimization algorithms and enhanced sampling schemes in molecular dynamics.

Improving the Convergence Rates for the Kinetic Fokker-Planck Equation by Optimal Control

“…where in the last step we referred to (8). Using density of C ∞ 0 (R 2d ) in V , we conclude that (29) holds for all y ∈ V and thus V -Y coercivity of a ε follows.…”

“…This motivates the introduction of controls into (1) as it has been discussed in, e.g., [8,11,27]. One of the main challenges in such a control framework is to ensure that the invariant measure of the uncontrolled dynamics (1) is unaltered.…”

Poisson Equations with locally-Lipschitz coefficients and Uniform in Time Averaging for Stochastic Differential Equations via Strong Exponential Stability

“…In conclusion, if (46) holds, then we may apply Theorem 3.2 to ( 35)-( 36) so there exists C > 0, such that…”

“…Averaging methods are extremely effective in applications, see e.g. [8,17,21,33,46], with no claim to completeness of references; yet, a long-standing criticism of such techniques is the following: while one can typically only prove that the averaged dynamics is a good approximation of the original slow-fast system for finite-time windows (with estimates that deteriorate in time), the averaged dynamics is often used in practice to make predictions about the long-time behaviour of the slow-fast system. There is however plenty of numerical evidence that the slow-fast system does often converge to the averaged dynamics uniformly in time (i.e.…”

“…Hence in Section 3 we first state all our results in the non-fully coupled case and then in the fully coupled one and the whole paper refers to the non-fully coupled case, with the exception of Section 3.2 and Section 6.3, where the fully coupled regime is treated. Because in many cases of interest in applications the diffusion coefficients are constant, see [8,23,46] and references therein (hence falling in the non-fully coupled case) in the non-fully coupled regime we express all our theorems in terms of directly verifiable conditions on the coefficients of (1)-(2) (not just in terms of SES requirements). Similarly, the techniques of this paper can be used, without requiring any new ideas, to study the homogenization regime and the case in which W t and B t are not independent of each other.…”