Markov Chain Importance Sampling -- a highly efficient estimator for MCMC

Abstract

Markov chain algorithms are ubiquitous in machine learning and statistics and many other disciplines. In this work we present a novel estimator applicable to several classes of Markov chains, dubbed Markov chain importance sampling (MCIS). For a broad class of Metropolis-Hastings algorithms, MCIS efficiently makes use of rejected proposals. For discretized Langevin diffusions, it provides a novel way of correcting the discretization error. Our estimator satisfies a central limit theorem and improves on error per CPU cycle, often to a large extent. As a by-product it enables estimating the normalizing constant, an important quantity in Bayesian machine learning and statistics.An efficient estimation of ρ θ for two classes of Markov chain algorithms, the Euler-Maruyama discretized Langevin diffusion (DL) and Metropolis-Hastings (MH), results in our integral estimators S MHIS K and S DLIS K , which strongly outperform the standard estimator. In addition, our estimators allow to approximate the normalizing constant Z of the target density, a quantity that is important for Bayesian model selection and averaging. This refutes the folk theorem that Metropolis-Hastings algorithms do not allow for an easy approximation of the normalizing constant Z. The paper is structured as follows. In Section 2, we introduce Markov chain importance sampling (MCIS) and derive explicit formulas for and approximations of ρ θ for MH and DL. Section 3 deals with convergence properties of S MHIS K , and a law of large numbers as well as a central limit theorem Preprint. Work in progress.

On the use of approximate Bayesian computation Markov chain Monte Carlo with inflated tolerance and post-correction

“…Namely, as in the proof of Theorem 4, we find that (Θ k , Y k , Θk+1 , Y k+1 ) k≥1 is a Harris recurrent Markov chain with invariant distribution πδ (θ, y, θ, ỹ) = πδ (θ, y)q(θ, y; θ, ỹ), and π (θ, y, θ, ỹ)/π δ (θ, y, θ, ỹ) = c w δ, (y), where q(θ, y; θ , y ) = q(θ, θ )g(y | θ ). Therefore, E WR δ, (f ) is a strongly consistent estimator of (Rudolf and Sprungk, 2018;Schuster and Klebanov, 2018) for alternative waste recycling estimators based on importance sampling analogues.…”

Compressed Monte Carlo with application in particle filtering

“…In the so-called layered adaptive importance sampling (LAIS) algorithm [33] and similar methods [46], an MCMC algorithm is used for obtaining a set of mean parameters {µ 1 , ..., µ T }. Then, one sample x t is drawn from a proposal density with mean µ t , i.e., x t ∼ q(x t |µ t , C) where C is a covariance matrix and t = 1, ...., T .…”

“…where a temporal mixture is used in the denominator [33], [46]. With this choice, very good performance can be obtained, but the computational cost of evaluating the weight denominator increases with T 2 [33].…”

“…Therefore, the C-PF is faster than a standard particle filter and is particularly convenient when the likelihood evaluation is costly. We also provide an example of C-MC in modern adaptive IS schemes to allow the use of expensive mixtures as the denominator of the importance weights [33], [46]. More details are provided in Section V. Finally, note that similar and related ideas have been presented in different works and several applications, such as diffusion estimation [8], [40], smoothing techniques [13], and as alternative resampling procedures in particle filtering [25], [26].…”

Marginal likelihood computation for model selection and hypothesis testing: an extensive review

“…Special case with recycling samples. The method in [84] can be considered as a special case of LAIS when N = 1, and {µ t = x t } i.e., all the samples {x t } T t=1 are generated by the unique MCMC chain with random walk proposal ϕ(x|x t−1 ) = q(x|x t−1 ) with invariant density π(x). In this scenario, the two layers of LAIS are collapsed in a unique layer, so that {µ t = x t }.…”