Improved convergence guarantees for learning Gaussian mixture models by EM and gradient EM

Abstract

We consider the problem of estimating the parameters a Gaussian Mixture Model with K components of known weights, all with an identity covariance matrix. We make two contributions. First, at the population level, we present a sharper analysis of the local convergence of EM and gradient EM, compared to previous works. Assuming a separation of Ω( √ log K), we prove convergence of both methods to the global optima from an initialization region larger than those of previous works. Specifically, the initial guess of each component can be as far as (almost) half its distance to the nearest Gaussian. This is essentially the largest possible contraction region. Our second contribution are improved sample size requirements for accurate estimation by EM and gradient EM. In previous works, the required number of samples had a quadratic dependence on the maximal separation between the K components, and the resulting error estimate increased linearly with this maximal separation. In this manuscript we show that both quantities depend only logarithmically on the maximal separation.

An Approximate Expectation-Maximization for Two-Dimensional Multi-Target Detection

“…where y k is given by (30) and A k is given by (31), and is the Hadamard product. Notably, we can rewrite (30) as (32), where q(p, , ω; , m, s) is given by (33). Furthermore, we can rewrite A k from (31) as (34), where g( , ω; , m, s; ˜ , m, s) is given by (35).…”

“…and the shifted and padded PSWF eigenfunctions ψ N,n , denoted as q(p, , ω; , m, s) (33), can be computed just once at the beginning of the algorithm with complexity O N 2 L 2 K 5 max . All in all, the computational complexity of computing…”

“…EM is an algorithm for finding a local maximum of a likelihood function with nuisance variables. It is widely used in many machine learning and statistics tasks, with applications to parameter estimation [32], mixture models [33], deep belief networks [34], and independent component analsyis [35], to name but a few. The EM algorithm was introduced to the cryo-EM community in [36], and is by now the most popular method for 3-D recovery from picked particles [29], where the 3-D rotations, but not the 2-D locations, are treated as nuisance variables.…”

Yarn apparent evenness detection based on L0 norm smoothing and the expectation maximization method

“…Most of the wrong division is caused by burrs on the yarn evenness. The EM algorithm [28][29][30] classifies the pixels according to the grayscale distribution of the image, and then divides the image according to the attributes of the class. This algorithm can not only segment the complete yarn evenly, but also preserves the details of the contour.…”