A new iterative initialization of EM algorithm for Gaussian mixture models

Abstract

Background The expectation maximization (EM) algorithm is a common tool for estimating the parameters of Gaussian mixture models (GMM). However, it is highly sensitive to initial value and easily gets trapped in a local optimum. Method To address these problems, a new iterative method of EM initialization (MRIPEM) is proposed in this paper. It incorporates the ideas of multiple restarts, iterations and clustering. In particular, the mean vector and covariance matrix of sample are calculated as the initial values of the iteration. Then, the optimal feature vector is selected from the candidate feature vectors by the maximum Mahalanobis distance as a new partition vector for clustering. The parameter values are renewed continuously according to the clustering results. Results To verify the applicability of the MRIPEM, we compared it with other two popular initialization methods on simulated and real datasets, respectively. The comparison results of the three stochastic algorithms indicate that MRIPEM algorithm is comparable in relatively high dimensions and high overlaps and significantly better in low dimensions and low overlaps.

Unsupervised metric learning via K-ISOMAP for high-dimensional data clustering

“…To find a maximum likelihood solution it is necessary to calculate the derivatives of the log-likelihood function with respect to all unknown parameters and latent variables and simultaneously solve all the equations obtained. In statistical models with latent variables, such as the GMM model, this is generally impossible [36].…”