Postprocessing of MCMC

Abstract

Markov chain Monte Carlo is the engine of modern Bayesian statistics, being used to approximate the posterior and derived quantities of interest. Despite this, the issue of how the output from a Markov chain is postprocessed and reported is often overlooked. Convergence diagnostics can be used to control bias via burn-in removal, but these do not account for (common) situations where a limited computational budget engenders a bias-variance trade-off. The aim of this article is to review state-of-the-art techniques for postprocessing Markov chain output. Our review covers methods based on discrepancy minimization, which directly address the bias-variance trade-off, as well as general-purpose control variate methods for approximating expected quantities of interest. Expected final online publication date for the Annual Review of Statistics and Its Application, Volume 9 is March 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Vector-Valued Control Variates

“…For this problem, we trained all CVs through stochastic optimisation and use m = (50, 50) MC samples. This synthetic example was originally used by South et al (2022) to show one of the drawbacks of kernel-based CVs, namely that the fitted CV will usually tend to β in parts of the domain where we do not have any function evaluations. This phenomenon can be observed on the red lines in Figure 2 (left and center) which gives a CV based on a squared-exponential kernel.…”

“…Furthermore, if g is chosen appropriately, the variance of f − g will be much smaller than that of f , and a smaller number of samples will be required for the estimator to attain a given level of accuracy. The reader is referred to Si et al (2021); South et al (2022) for two recent reviews of this literature.…”

“…Figure2: Numerical integration of problem fromSouth et al (2022). Left and center: Illustration of f 1 and f 2 , as well as the corresponding kernel-based CVs and vv-CVs obtained through stochastic optimisation when Π 1 = N (0, 1) and Π 2 = N (0, 1.25).…”

Meta-learning Control Variates: Variance Reduction with Limited Data

“…An effective control variate is one for which the difference f − g has smaller MC variance than f (or asymptotic variance, in the case of MCMC). CVs have proved successful in a range of challenging tasks in statistical physics [Assaraf and Caffarel, 1999], Bayesian statistics [Dellaportas and Kontoyiannis, 2012, Mira et al, 2013, Oates et al, 2017, South et al, 2022c, gradient estimation in variational inference [Grathwohl et al, 2018, Shi et al, 2022 and MCMC [Baker et al, 2019], reinforcement learning Liu et al [2018Liu et al [ , 2019, and computer graphics [Müller et al, 2020].…”

Kernel Stein Discrepancy thinning: a theoretical perspective of pathologies and a practical fix with regularization

“…On the other hand, thinning the Markov chain allows for compressing the MCMC output and may also reduce the correlation between the iteratively selected points. More recently, promising kernel-based procedures were proposed to automatically remove the burn-in period, compress the output, and reduce the asymptotic bias (South et al, 2022). These approaches consist in minimizing a kernel-based discrepancy measure D(P, Q m ) between the empirical distribution Q m of a subsample of the MCMC output of size m, and the target distribution P. In this respect, minimization of the maximum mean discrepancy (MMD) was investigated by several authors, but these strategies require the full knowledge of the target distribution P, whose density is not tractable in non-conjugate Bayesian inference.…”