Variance reduction for Markov chains with application to MCMC

Abstract

In this paper we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive feature of the proposed approach is its ability to significantly reduce the overall finite sample variance. This feature is theoretically demonstrated by means of a deep non asymptotic analysis of a variance reduced functional as well as by a thorough simulation study. In particular we apply our method to various MCMC Bayesian estimation problems where it favourably compares to the existing variance reduction approaches.Assume that P has the unique invariant distribution π, that is, X π(dx)P (x, dy) = π(dy). Under appropriate conditions, the Markov kernel P may be shown to converge to the stationary distribution π, that is, for any x ∈ X, lim n→∞ P n (x, ·) − π TV = 0,

Unbiased Markov Chain Monte Carlo Methods with Couplings

“…A control variate g should therefore be selected such that π ( g )=0 and σ ( h − g )≪ σ ( h ). In inequality (3.2) it was demonstrated that, in the large m and k limit, the quantity σ ( h ) 2 is just the asymptotic variance from traditional MCMC sampling; existing gradient‐based control variates can therefore be used (Belomestny et al ., ; Mijatović and Vogrinc, ). However, at finite m and k the dependence of σ ( h ) on h is far from explicit.…”

Semi-exact control functionals from Sard’s method

“…The main challenge in developing control variates, or functionals, based on Stein operators is therefore to find a function g n such that the asymptotic variance σ(f − f n ) 2 is small. To explicitly minimize asymptotic variance, Mijatović & Vogrinc (2018); Belomestny et al (2020) and Brosse et al (2019) restricted attention to particular Metropolis-Hastings or Langevin samplers for which asymptotic variance can be explicitly characterized. The minimization of empirical variance has also been proposed and studied in the case where samples are independent (Belomestny et al, 2017) and dependent (Belomestny et al, 2020(Belomestny et al, , 2019.…”

“…To explicitly minimize asymptotic variance, Mijatović & Vogrinc (2018); Belomestny et al (2020) and Brosse et al (2019) restricted attention to particular Metropolis-Hastings or Langevin samplers for which asymptotic variance can be explicitly characterized. The minimization of empirical variance has also been proposed and studied in the case where samples are independent (Belomestny et al, 2017) and dependent (Belomestny et al, 2020(Belomestny et al, , 2019. For an approach that is not tied to a particular Markov kernel, authors such as Assaraf & Caffarel (1999) and Mira et al (2013) proposed to minimize mean squared error along the sample path, which corresponds to the case of an independent sampling method.…”

“…In the Bayesian context, Assaraf & Caffarel (1999); Mira et al (2013) and Oates et al (2017) addressed this challenge by using f n = c n + Lg n where c n ∈ R, g n is a user-chosen parametric or nonparametric function and L is an operator, for example the Langevin Stein operator (Stein, 1972;Gorham & Mackey, 2015), that depends on p through its gradient and satisfies (Lg n )(x)p(x)dx = 0 under regularity conditions (see Lemma 1). Convergence of I CV (f ) to I(f ) has been studied under (strong) regularity conditions and, in particular (i) if g n is chosen parametrically, then in general lim inf σ(f − f n ) 2 > 0 so that, even if asymptotic variance is reduced, convergence rates are unaffected; (ii) if g n is chosen in an appropriate non-parametric manner then lim sup σ(f − f n ) 2 = 0 and a smaller number O( −2+δ ), 0 < δ < 2, of calls to f , p and its gradient are required to achieve an approximation error of O P ( ) for the integral (see Oates et al, 2019;Mijatović & Vogrinc, 2018;Barp et al, 2018;Belomestny et al, 2017Belomestny et al, , 2019Belomestny et al, , 2020. In the parametric case Lg n is called a control variate while in the non-parametric case it is called a control functional.…”

Reduced variance analysis of molecular dynamics simulations by linear combination of estimators

“…Recent promising examples of such endeavours include new strategies for the determination of generic observables [9][10][11] and transport properties [12][13][14] , as well as force-based estimators to sample local properties such as number, charge and polarization densities, radial distribution functions (RDF) or local transport properties with a reduced variance 10,11,[15][16][17][18][19][20][21] . The availability of different estimators for the same observable opens the possibility of exploiting another well known approach to further reduce the variance, namely the control variates method [22][23][24][25][26][27][28][29][30] . Here we show the potential of combining of estimators for the determination of RDF and onedimensional density profiles of a bulk and confined fluid, dea) Electronic mail: benjamin.rotenberg@sorbonne-universite.fr fined (for a one-component system of N particles in a volume V ) respectively as the ensemble averages:…”