On Convergence Properties of the EM Algorithm for Gaussian Mixtures

Abstract

We build up the mathematical connection between the “Expectation-Maximization” (EM) algorithm and gradient-based approaches for maximum likelihood learning of finite gaussian mixtures. We show that the EM step in parameter space is obtained from the gradient via a projection matrix P, and we provide an explicit expression for the matrix. We then analyze the convergence of EM in terms of special properties of P and provide new results analyzing the effect that P has on the likelihood surface. Based on these mathematical results, we present a comparative discussion of the advantages and disadvantages of EM and other algorithms for the learning of gaussian mixture models.

On the Number of Modes of a Gaussian Mixture

“…This is similar to a result from Xu and Jordan (1996) for the EM algorithm when used to estimate the parameters π m , µ m and Σ m from a data sample.…”

A Maximum Profit Coverage Algorithm with Application to Small Molecules Cluster Identification

“…MEM is based on the common method of likelihood maximization via Expectation Maximization, i.e. the EM algorithm [17,21,25]. See a description of the algorithm in Appendix D. The input to the EM algorithm includes the number of distributions k and will return a maximum likelihood estimator of the parameters of k distributions that best explain S. In order to find the optimal k we use the rule given by [22] to estimate the number of generating distributions, i.e k that maximizes max log like(k) + k log(N ), where N is the number of samples, and max log like(k) is the maximal value of the likelihood function for k distributions.…”

Automatic seed initialization for the expectation-maximization algorithm and its application in 3D medical imaging

“…The EM algorithm shows robust and repeatable performance in the segmentations of heart, brain and abdominal images. The EM algorithm is locally convergent [6,14,15] so we have introduced an automatic seeding method that uses local maxima in the intensity histogram. The results are compared against a manual initialization, achieved by first manually selecting a region and then measuring the mean intensity values and variance in that region.…”

“…It is important to note that local convergence of the EM algorithm is assured [6,14,15]. The updates for the parameters for the GMM are the mixture values a m and the parameters of the Gaussian distributions y m ¼ {m m , s m }.…”