Stochastic gradient Hamiltonian Monte Carlo with variance reduction for Bayesian inference

Abstract

Gradient-based Monte Carlo sampling algorithms, like Langevin dynamics and Hamiltonian Monte Carlo, are important methods for Bayesian inference. In large-scale settings, full-gradients are not affordable and thus stochastic gradients evaluated on mini-batches are used as a replacement. In order to reduce the high variance of noisy stochastic gradients, Dubey et al. [2016] applied the standard variance reduction technique on stochastic gradient Langevin dynamics and obtained both theoretical and experimental improvements. In this paper, we apply the variance reduction tricks on Hamiltonian Monte Carlo and achieve better theoretical convergence results compared with the variance-reduced Langevin dynamics. Moreover, we apply the symmetric splitting scheme in our variance-reduced Hamiltonian Monte Carlo algorithms to further improve the theoretical results. The experimental results are also consistent with the theoretical results. As our experiment shows, variance-reduced Hamiltonian Monte Carlo demonstrates better performance than variance-reduced Langevin dynamics in Bayesian regression and classification tasks on real-world datasets. † denotes equal contribution

SHPOS: A Theoretical Guaranteed Accelerated Particle Optimization Sampling Method

“…Sampling from a Bayesian posterior distribution lies at the core of many modern machine learning tasks, such as topic modelling [Gan et al, 2015], reinforcement learning [Liu et al, 2017], and Bayesian neural networks [Hernández-Lobato and Adams, 2015]. Particle based Variational Inference (ParVI) methods have recently drawn great attention due to their empirical success in approximating the target posterior distribution [Liu and Wang, 2016;Liu et al, 2017;Feng et al, 2017;Liu and Zhu, 2018]. Typically, these methods update a finite set of interacting particles deterministically to approximately simulate infinite-particle gradient flows on the Wasserstein space P 2 (X ).…”

“…One representative method of this type is the Stein Variational Gradient Descent (SVGD) method [Liu and Wang, 2016], which updates the particles according to a gradient flow described by the Vlasov equation Braun and Hepp, 1977]. Subsequently, by exploiting the Riemannian structure of the Wasserstein space P 2 (X ), [Liu et al, 2019] proposed a Nesterov's-acceleration variant of SVGD called SVGD Wasserstein Nesterov's method (SVGD-WNes).…”

“…However, ParVI methods have an intractable pitfall that particles tend to collapse under certain condition due to the deterministic-update fashion with a limited number of particles [Zhang et al, 2020a;Zhuo et al, 2018], which indicates a large deviation of the particles' empirical distribution to target distribution. To understand the influence of finiteparticle approximation to the infinite-particle gradient flows, Liu et al [2019] provided a unified theory on the approximation property of different ParVI methods. They show that existing finite-particle ParVIs can be regarded as essentially smoothing operations on gradient flows, in the form of either smoothing the density or smoothing functions.…”

“…Inspired by the random exploration in the dynamics-based Markov Chain Monte Carlo methods, recent researches relieve the particle-collapsing phenomenon by introducing additional stochasticity into ParVI methods [Zhang et al, 2020a;Zhang et al, 2020b]. Zhang et al [2020a] integrated the gradient flow of the first-order Overdamped Langevin Dynamics (OLD) into the Vlasov equation used in SVGD, and proposed a new method called Stochastic Particle Optimization Sampling (SPOS). Specifically, SPOS updates a set of particles following the same mechanism as in SVGD with an additional drift term and an extra Gaussian random noise induced by OLD.…”

“…Compared with the first-order OLD, ULD possesses an auxiliary momentum term, and can be regarded as an accelerated secondorder dynamics of OLD. Typically, ULD based methods have both better theoretical guarantees and more competitive practical performance than their OLD based counterparts [Chen et al, 2014;Cheng et al, 2018;Zou et al, 2018;Zou et al, 2019]. We utilize a variant of the first order exponential integrator scheme to discretize the corresponding stochastic differential equation of the augmented flow, which is obtained by integrating the gradient flow of ULD into the Vlasov flow.…”