MCMC for Normalized Random Measure Mixture Models

Abstract

This paper concerns the use of Markov chain Monte Carlo methods for posterior sampling in Bayesian nonparametric mixture models with normalized random measure priors. Making use of some recent posterior characterizations for the class of normalized random measures, we propose novel Markov chain Monte Carlo methods of both marginal type and conditional type. The proposed marginal samplers are generalizations of Neal's well-regarded Algorithm 8 for Dirichlet process mixture models, whereas the conditional sampler is a variation of those recently introduced in the literature. For both the marginal and conditional methods, we consider as a running example a mixture model with an underlying normalized generalized Gamma process prior, and describe comparative simulation results demonstrating the efficacies of the proposed methods.Comment: Published in at http://dx.doi.org/10.1214/13-STS422 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

A Marginal Sampler for σ-Stable Poisson–Kingman Mixture Models

“…Marginal MCMC methods remove the infinite dimensionality of the problem by exploiting the tractable marginalization with respect to the Dirichlet process. See Escobar (1994), MacEachern (1994) and Escobar & West (1995) , Barrios et al (2013), Favaro & Teh (2013) and Favaro et al (2014) for details.…”

“…We describe a non-conjugate case where the component parameters pYk q kPrKs cannot be marginalized out, and we derive an extension of Favaro & Teh (2013). In the case where the base distribution H 0 is conjugate to the observation's distribution Fp¨q, the component parameters can be marginalized out as well, which leads to an extension to Algorithm 3 of Neal (2000).…”

“…The Gibbs updates to the partition involve updating the cluster assignment of one observation, say the ith one, at a time. We can adapt the Reuse algorithm of Favaro & Teh (2013) to update our partitions.…”

Density Estimation: Models Beyond the DP

“…Two recent papers (Barrios et al 2013;Favaro and Teh 2013) discuss mixture models with an NRMI prior on the mixing measure, similar to (7.5), but with any other NRMI prior replacing the specific DP prior. Both discuss the specific case of the normalized generalized Gaussian process (NGGP), which is attractive as it includes the DP as well as several other examples as special cases.…”

A blocked Gibbs sampler for NGG-mixture models via a priori truncation

“…the random probability), resorting to generalized Polya urn schemes (MacEachern, 1998); see Neal (2000) for a review on the subject. Recently, Favaro and Teh (2013) developed algorithms of both types for mixture models with NRMI mixing measures.…”