Mixture Models With a Prior on the Number of Components

Abstract

A natural Bayesian approach for mixture models with an unknown number of components is to take the usual finite mixture model with symmetric Dirichlet weights, and put a prior on the number of components—that is, to use a mixture of finite mixtures (MFM). The most commonly-used method of inference for MFMs is reversible jump Markov chain Monte Carlo, but it can be nontrivial to design good reversible jump moves, especially in high-dimensional spaces. Meanwhile, there are samplers for Dirichlet process mixture (DPM) models that are relatively simple and are easily adapted to new applications. It turns out that, in fact, many of the essential properties of DPMs are also exhibited by MFMs—an exchangeable partition distribution, restaurant process, random measure representation, and stick-breaking representation—and crucially, the MFM analogues are simple enough that they can be used much like the corresponding DPM properties. Consequently, many of the powerful methods developed for inference in DPMs can be directly applied to MFMs as well; this simplifies the implementation of MFMs and can substantially improve mixing. We illustrate with real and simulated data, including high-dimensional gene expression data used to discriminate cancer subtypes.

On a loss-based prior for the number of components in mixture models

“…A well-known and widely used method is the reversible Markov chain Monte Carlo (Green, 1995) which, due to its non-trivial set up, has led to the search of alternatives. A recent and interesting one is proposed by Miller and Harrison (2018), where the model in (1) is written as…”

“…Throughout the paper we will adopt, for the weights and the component parameters, the priors proposed in Miller and Harrison (2018); this will not affect the analysis of the results and the comparisons among different priors for k.…”

Bayesian Double Feature Allocation for Phenotyping With Electronic Health Records

“…Clustering models partition entities into mutually exclusive latent groups (clusters). Numerous methods have been developed, including algorithm-based approaches such as kmeans, and model-based clustering methods such as finite mixture models (Richardson & Green, 1997;Miller & Harrison, 2018) and infinite mixture models (Lau & Green, 2007;Favaro & Teh, 2013;Barrios et al , 2013, for example). Partitioning symptoms, similar to graphical models, may discover latent diseases that are related to subsets of symptoms whereas clustering patients suggests latent diseases that are shared among groups of patients.…”

Estimating Autoantibody Signatures To Detect Autoimmune Disease Patient Subsets

“…We can either add another hierarchy for Bayesian dewarping or develop regression models for {λ * } to incorporate disease phenotype information or other covariates, e.g, age and gender to refine disease subsetting. Second, multiple autoantibodies produced against a particular Z or extracted continuous intensity shape information for each landmark and lane either by regularization or using shrinkage priors in a Bayesian framework for encouraging few and maximally different complexes (e.g., Broderick and others, 2013;Miller and Harrison, 2015). Our preliminary results (not shown here) show good subset and signature estimation performance.…”