Statistical convergence of the EM algorithm on Gaussian mixture models

Zhao, Ruofei; Li, Yuanzhi; Sun, Yuekai

doi:10.1214/19-ejs1660

Cited by 25 publications

(38 citation statements)

References 26 publications

(41 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Let

denote the ground truth means of

Gaussian components in a

-dimensional Gaussian mixture dataset and let

where

and

denote the ground truth means of component

and

, denote the minimum distance between the means of any two components. Zhao et al (2020) provides a convergence guarantee of the EM algorithm to the global optima, as long as the following condition is true:

…”

Section: Methodsmentioning

confidence: 99%

Artificial-cell-type aware cell-type classification in CITE-seq

Lian

Xin

et al. 2020

Bioinformatics

View full text Add to dashboard Cite

Motivation Cellular Indexing of Transcriptomes and Epitopes by sequencing (CITE-seq), couples the measurement of surface marker proteins with simultaneous sequencing of mRNA at single cell level, which brings accurate cell surface phenotyping to single-cell transcriptomics. Unfortunately, multiplets in CITE-seq datasets create artificial cell types (ACT) and complicate the automation of cell surface phenotyping. Results We propose CITE-sort, an artificial-cell-type aware surface marker clustering method for CITE-seq. CITE-sort is aware of and is robust to multiplet-induced ACT. We benchmarked CITE-sort with real and simulated CITE-seq datasets and compared CITE-sort against canonical clustering methods. We show that CITE-sort produces the best clustering performance across the board. CITE-sort not only accurately identifies real biological cell types (BCT) but also consistently and reliably separates multiplet-induced artificial-cell-type droplet clusters from real BCT droplet clusters. In addition, CITE-sort organizes its clustering process with a binary tree, which facilitates easy interpretation and verification of its clustering result and simplifies cell-type annotation with domain knowledge in CITE-seq. Availability and implementation http://github.com/QiuyuLian/CITE-sort. Supplementary information Supplementary data is available at Bioinformatics online.

show abstract

“…Let

denote the ground truth means of

Gaussian components in a

-dimensional Gaussian mixture dataset and let

where

and

denote the ground truth means of component

and

…”

Section: Methodsmentioning

confidence: 99%

Artificial-cell-type aware cell-type classification in CITE-seq

Lian

Xin

et al. 2020

Bioinformatics

View full text Add to dashboard Cite

show abstract

“…Currently, the best known sample requirements for EM are n =Ω(K 3 dR 2 max /R 2 min ), whereas for gradient EM, n =Ω(K 6 dR 6 max /R 2 min ). In addition, the bounds for the resulting errors increase linearly with R max , see [22,23]. Note that in these two results, the required number of samples increases at least quadratically with the maximal separation between clusters, even though increasing R max should make the problem easier.…”

Section: Introductionmentioning

confidence: 93%

“…Let us compare our results to several recent works that derived convergence guarantees for EM and gradient EM. [23] and [22] proved local convergence to the global optima under a much larger minimal separation of R min ≥ C min(d, K) log K. In addition, the requirement on the initial estimates had a dependence on the maximal separation, μ i − μ * i ≤ 1 2 R min − C 1 min(d, K) log max(R max , K 3 ) for a universal constant C 1 . These results were significantly improved by [13], who proved the local convergence of the EM algorithm for the more general case of spherical Gaussians with unknown weights and variances.…”

Section: Introductionmentioning

confidence: 99%

Improved convergence guarantees for learning Gaussian mixture models by EM and gradient EM

Segol

Nadler

2021

Electron. J. Statist.

View full text Add to dashboard Cite

We consider the problem of estimating the parameters a Gaussian Mixture Model with K components of known weights, all with an identity covariance matrix. We make two contributions. First, at the population level, we present a sharper analysis of the local convergence of EM and gradient EM, compared to previous works. Assuming a separation of Ω( √ log K), we prove convergence of both methods to the global optima from an initialization region larger than those of previous works. Specifically, the initial guess of each component can be as far as (almost) half its distance to the nearest Gaussian. This is essentially the largest possible contraction region. Our second contribution are improved sample size requirements for accurate estimation by EM and gradient EM. In previous works, the required number of samples had a quadratic dependence on the maximal separation between the K components, and the resulting error estimate increased linearly with this maximal separation. In this manuscript we show that both quantities depend only logarithmically on the maximal separation.

show abstract

“…The convergence of the EM algorithm under Gaussian mixture model is well studied and therefore we can impose some alternative assumptions to substitute condition (A1) under this case. For example, if the conditions in Theorem 3.3 of Zhao et al [47] hold, the EM algorithm in Examples 2 and 3 converges to the MLE.…”

Section: Asymptoticsmentioning

confidence: 99%

Envelope method with ignorable missing data

Liu

Yang

2021

Electron. J. Statist.

View full text Add to dashboard Cite

Envelope method was recently proposed as a method to reduce the dimension of responses in multivariate regressions. However, when there exists missing data, the envelope method using the complete case observations may lead to biased and inefficient results. In this paper, we generalize the envelope estimation when the predictors and/or the responses are missing at random. Specifically, we incorporate the envelope structure in the expectation-maximization (EM) algorithm. As the parameters under the envelope method are not pointwise identifiable, the EM algorithm for the envelope method was not straightforward and requires a special decomposition. Our method is guaranteed to be more efficient, or at least as efficient as, the standard EM algorithm. Moreover, our method has the potential to outperform the full data MLE. We give asymptotic properties of our method under both normal and non-normal cases. The efficiency gain over the standard EM is confirmed in simulation studies and in an application to the Chronic Renal Insufficiency Cohort (CRIC) study.

show abstract

Statistical convergence of the EM algorithm on Gaussian mixture models

Cited by 25 publications

References 26 publications

Artificial-cell-type aware cell-type classification in CITE-seq

Artificial-cell-type aware cell-type classification in CITE-seq

Improved convergence guarantees for learning Gaussian mixture models by EM and gradient EM

Envelope method with ignorable missing data

Contact Info

Product

Resources

About