Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Acceleration

Abstract

Stochastic gradient Hamiltonian Monte Carlo (SGHMC) is a variant of stochastic gradient with momentum where a controlled and properly scaled Gaussian noise is added to the stochastic gradients to steer the iterates towards a global minimum. Many works reported its empirical success in practice for solving stochastic non-convex optimization problems, in particular it has been observed to outperform overdamped Langevin Monte Carlo-based methods such as stochastic gradient Langevin dynamics (SGLD) in many applications. Although asymptotic global convergence properties of SGHMC are well known, its finite-time performance is not well-understood. In this work, we study two variants of SGHMC based on two alternative discretizations of the underdamped Langevin diffusion. We provide finite-time performance bounds for the global convergence of both SGHMC variants for solving stochastic non-convex optimization problems with explicit constants. Our results lead to non-asymptotic guarantees for both population and empirical risk minimization problems. For a fixed target accuracy level, on a class of non-convex problems, we obtain complexity bounds for SGHMC that can be tighter than those for SGLD. These results show that acceleration with momentum is possible in the context of global non-convex optimization.

Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Learning

“…Our work continues these lines of research, the most similar setting to ours is the recent paper [GGZ18]. We summarize our contributions below:…”

“…By introducing the auxiliary SDEs (13), ( 14), we are able to achieve the rate (δ 1/4 + λ 1/4 ), see Theorem 2.8 for the case p = 2. This upper bound is better in the number of iterations and hence, improves Lemma 10 of [GGZ18]. Our analysis for variance of the algorithm is also different.…”

“…• If we consider the p-Wasserstein distance for 1 < p ≤ 2, in particular, when p → 1, Theorem 2.9 gives tighter bounds, compared to Theorem 2 of [GGZ18].…”

“…• Diffusion approximation. In Lemma 10 of [GGZ18], the upper bound for the 2-Wasserstein distance between the SGHMC algorithm at step k and underdamped SDE at time t = kλ is (up to constants) given by (δ 1/4 + λ 1/4 ) √ kλ log(kλ), which depends on the number of iteration k. Therefore obtaining a precision ε requires a careful choice of k, λ and even kλ. By introducing the auxiliary SDEs (13), ( 14), we are able to achieve the rate (δ 1/4 + λ 1/4 ), see Theorem 2.8 for the case p = 2.…”

“…Each term in Ξ n has an error of order δ, the total variance in Ξ n is of order T δ. However, unlike [RRT17], [GGZ18], our technique does not accumulate variance errors over time, as shown in (30). Recently in [BCM + 18], the authors imposed no condition for variance of the estimated gradient, but employ the conditional L-mixing property of data stream, and hence variance is controlled by the decay of mixing property, see their Lemma 8.6.…”