Convergence of latent mixing measures in finite and infinite mixture models

Abstract

This paper studies convergence behavior of latent mixing measures that arise in finite and infinite mixture models, using transportation distances (i.e., Wasserstein metrics). The relationship between Wasserstein distances on the space of mixing measures and f -divergence functionals such as Hellinger and Kullback-Leibler distances on the space of mixture distributions is investigated in detail using various identifiability conditions. Convergence in Wasserstein metrics for discrete measures implies convergence of individual atoms that provide support for the measures, thereby providing a natural interpretation of convergence of clusters in clustering applications where mixture models are typically employed. Convergence rates of posterior distributions for latent mixing measures are established, for both finite mixtures of multivariate distributions and infinite mixtures based on the Dirichlet process.

Mixture Models With a Prior on the Number of Components

“…implies G = G ′.) While Nguyen (2013) has shown that under certain conditions, DPMs are consistent for the mixing distribution (in the Wasserstein metric), Miller and Harrison (2013, 2014) have shown that the DPM posterior on the number of clusters is typically not consistent for the number of components, at least when the concentration parameter is fixed; the question remains open when using a prior on the concentration parameter, but we conjecture that it is still not consistent. On the other hand, MFMs are consistent for the mixing distribution and the number of components (for Lebesgue almost-all values of the true parameters) under very general conditions (Nobile, 1994); this is a straightforward consequence of Doob’s theorem.…”

Posterior contraction of the population polytope in finite admixture models

“…Set ε = ε 1 , the conclusion then follows by invoking Equation (12). The scenario of (ii) proceeds in the same way.…”

“…This framework continues to be very useful, but the analysis of mixing measure estimation in multi-level models presents distinct new challenges. In Section 4, we shall formulate an abstract theorem (Theorem 4) on posterior contraction of latent variables of interest in an admixture model, given m × n data, by reposing on the framework of [7] (see also [12]). The main novelty here is that we work on the space of latent variables (e.g., space of latent population structures endowed with Hausdorff or a comparable metric) as opposed to the space of data densities endowed with Hellinger metric.…”

“…Chen provided the optimal convergence rate of mixing measures in several finite mixtures for univariate data [5] (see also [10]). Recent progress on multivariate mixture models include papers by Rousseau and Mengersen [14] and Nguyen [12]. In [12], posterior contraction rates of mixing measures in several finite and infinite mixture models for multivariate and continuous data were obtained.…”

“…Recent progress on multivariate mixture models include papers by Rousseau and Mengersen [14] and Nguyen [12]. In [12], posterior contraction rates of mixing measures in several finite and infinite mixture models for multivariate and continuous data were obtained. Toussile and Gassiat established consistency of a penalized MLE procedure for a finite admixture model [17].…”