An EM-Like Algorithm for Semi- and Nonparametric Estimation in Multivariate Mixtures

Abstract

Abstract:We propose an algorithm for nonparametric estimation for finite mixtures of multivariate random vectors that strongly resembles a true EM algorithm. The vectors are assumed to have independent coordinates conditional upon knowing which mixture component from which they come, but otherwise their density functions are completely unspecified. Sometimes, the density functions may be partially specified by Euclidean parameters, a case we call semiparametric. Our algorithm is much more flexible and easily applicable than existing algorithms in the literature; it can be extended to any number of mixture components and any number of vector coordinates of the multivariate observations. Thus it may be applied even in situations where the model is not identifiable, so care is called for when using it in situations for which identifiability is difficult to establish conclusively. Our algorithm yields much smaller mean integrated squared errors than an alternative algorithm in a simulation study. In another example using a real dataset, it provides new insights that extend previous analyses. Finally, we present two different variations of our algorithm, one stochastic and one deterministic, and find anecdotal evidence that there is not a great deal of difference between the performance of these two variants.

Model based labeling for mixture models

“…For the semi-parametric model, similar to Bordes, Chauveau, and Vandekerkhove (2007) and Benaglia, Chauveau, and Hunter (2009), the convergence property of the Algorithm 2 has not been established and needs further research, although empirically, the Algorithm 2 worked quite well and did converge for all the data sets we tried. One might also use one-step of Algorithm 2 to speed up the MBLSP method if one starts the Algorithm 2 from some good labels (such as NORMLH labels (Yao and Lindsay 2009)).…”

“…We will call such mixture model a semi-parametric mixture model due to the symmetric restriction (equal mixing proportions and permutation symmetric component density functions). By extending the semi-parametric EM algorithm proposed by Bordes, Chauveau, and Vandekerkhove (2007) and Benaglia, Chauveau, and Hunter (2009), we propose the following EM-like algorithm to fit the model (2).…”

“…As suggested by Benaglia, Chauveau, and Hunter (2009), one might also use an iterative procedure in which the value of H is modified after the new labels. For simplicity, in all of our examples in this section, we used the bandwidth (10) All the computations were done in Matlab 7.0 using a personal desktop with Intel Core 2 Quad CPU 2.40GHz.…”

Economics at the FTC: Fraud, Mergers and Exclusion

“…The main assumption required for non-parametric identification is that the signals are independent conditional on type. 20 We applied the statistical algorithms of Benaglia et al (2009a) and Levine et al (2011) to estimate the non-parametric finite mixture model; both are implemented in R package mixtools (Benaglia et al 2009b). The main algorithm that we used estimates the mixture model through an EM-like approach.…”

Nonparametric spectral-based estimation of latent structures

“…Given identification, some authors have set out to develop methods to estimate latent structures. Benaglia, Chauveau and Hunter [2009] and Levine, Hunter and Chauveau [2011] have constructed EM-like algorithms for finite mixtures. Gassiat, Cleynen and Robin [2013] investigate the use of penalized-likelihood methods in hidden Markov models.…”