Using a Markov Chain to Construct a Tractable Approximation of an Intractable Probability Distribution

Abstract

Let π denote an intractable probability distribution that we would like to explore. Suppose that we have a positive recurrent, irreducible Markov chain that satisfies a minorization condition and has π as its invariant measure. We provide a method of using simulations from the Markov chain to construct a statistical estimate of π from which it is straightforward to sample. We show that this estimate is "strongly consistent" in the sense that the total variation distance between the estimate and π converges to 0 almost surely as the number of simulations grows. Moreover, we use some recently developed asymptotic results to provide guidance as to how much simulation is necessary. Draws from the estimate can be used to approximate features of π or as intelligent starting values for the original Markov chain. We illustrate our methods with two examples.2

Fixed-Width Output Analysis for Markov Chain Monte Carlo

“…This is the approach we use in our simulations. Further practical advice on simulating the split chain is given in Geyer and Thompson (1995), Hobert et al (2002), Hobert et al (2003) and Jones andHobert (2001, 2004).…”

Exact sampling for intractable probability distributions via a Bernoulli factory

“…The exact sampling algorithm considered here utilizes a mixture representation for π (Asmussen et al, 1992;Hobert et al, 2006;Hobert and Robert, 2004), however, we must first develop the split chain. The main assumption necessary is that X satisfies a one-step minorization condition, i.e.…”

“…The representation at (4) can be obtained from results in Asmussen et al (1992) by applying their methods to the split chain. Alternatively, Hobert and Robert (2004) obtain the representation when s(x) has the specific form εI C (x) and Hobert et al (2006) obtain the representation with the more general minorization shown here.…”

“…This approach has never been successfully implemented, however, it has been used to obtain approximate draws from π, see for example Blanchet and Thomas (2007), Hobert et al (2006) and Hobert and Robert (2004). The difficult part is drawing from the discrete distribution corresponding to the mixture weights, which is done via a rejection sampling approach.…”

Markov chain Monte Carlo estimation of quantiles

“…, which have been implemented in several practically relevant statistical models; see e.g Doss and Tan (2013)Gilks et al (1998);Hobert et al (2006);Jones et al (2006);Jones and Hobert (2001);Roy and Hobert (2007).…”