Direct simulation for discrete mixture distributions

Abstract

We demonstrate how to perform direct simulation for discrete mixture models. The approach is based on directly calculating the posterior distribution using a set of recursions which are similar to those of the Forward-Backward algorithm. Our approach is more practicable than existing perfect simulation methods for mixtures. For example, we analyse 1096 observations from a 2 component Poisson mixture, and 240 observations under a 3 component Poisson mixture (with unknown mixture proportions and Poisson means in each case). Simulating samples of 10,000 perfect realisations took about 17 minutes and an hour respectively on a 900 MHz ultraSPARC computer. Our method can also be used to perform perfect simulation from Markov-dependent mixture models. A byproduct of our approach is that the evidence of our assumed models can be calculated, which enables different models to be compared.

Exact Bayesian inference via data augmentation

“…That is, π(θ i |θ i− , y, x) = π(θ i |y, x). This is seen, for example, in simple Normal mixture models with known variance (Diebolt and Robert 1994) or Poisson mixture models (Fearnhead 2005) and for the infection and recovery parameters of the general stochastic epidemic model with unknown infection times (y) and observed recovery times (x), see for example, O'Neill and Roberts (1999), Neal and Roberts (2005) and Kypraios (2007). Thus, in this case,…”

“…Collapsing can equally be applied to any Metropolis-Hastings algorithm, see, for example, Neal and Roberts (2005) and Kypraios (2007). In Fearnhead (2005) and Fearnhead (2006), the idea of collapsing is taken a stage further in the case where the number of possibilities of y is finite by computing π(y|x) exactly, and hence, express the posterior distribution, π(θ |x), as a finite mixture distribution. Specifically, Fearnhead (2005) and Fearnhead (2006) consider mixture models and change-point models, respectively, with the main focus of both papers perfect simulation from the posterior distribution as an alternative to MCMC.…”

“…The generic framework covers mixture models (Fearnhead 2005) and change-point models (Fearnhead 2006) as important special cases, but is applicable to a wide range of problems including two-level mixing Reed-Frost epidemic model (Sect. 3) and integer valued autoregressive time series models (Sect.…”

“…2, we show how in general y can be chosen and how sufficient statistics can be exploited to extend the usefulness of the methodology. Thus we extend the methodology introduced in Fearnhead (2005) beyond mixture models by identifying the key features which make (1.3) practical to use. A natural alternative to computing the exact posterior distribution is to use MCMC, and in particular, the Gibbs sampler, to obtain a sample from the posterior distribution.…”

A new rejection sampling method without using hat function

“…Note that for certain simpler versions of (34), exact simulation from its posterior is readily available. For example for the simple mixture models with known , methods in [19] and [13] and the method of adaptive rejection sampling for log-concave densities base on the algorithm in [21] can draw exact realisations from its posterior. The application of all these methods is limited to small sample sizes and small number of components.…”