Estimating the number of components in finite mixture models via the Group-Sort-Fuse procedure

Manole, Tudor; Khalili, Abbas

doi:10.1214/21-aos2072

Cited by 12 publications

(16 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Theorem 2(ii) also follows by the same proof technique as Theorem 7.4 of van de Geer (2000), with modifications to account for the presence of the penalty in the definition of G n . An analogue of this result was previously proven by Manole and Khalili (2021), though with different conditions on the tuning parameter ξ n . For completeness, we provide a self-contained proof of Theorem 2(ii), under the conditions on ξ n required for our development.…”

Section: A Proofssupporting

confidence: 70%

“…Its application to finite mixture models has previously been discussed by Ho and Nguyen (2016b), who also argue that condition B(k) is satisfied by a broad collection of parametric families F, including the multivariate location-scale Gaussian and Student-t families. A version of Theorem 2(ii) is implicit in the work of Manole and Khalili (2021), though with a stronger condition on the tuning parameter ξ n . We provide a self-contained proof of this result in Appendix A for completeness.…”

Section: Convergence Rates For Maximum Likelihood Density Estimatorsmentioning

confidence: 99%

“…We implement the penalized MLE G n using Algorithm 1, which is a slight modification of the EM algorithm Dempster et al (1977) accounting for the penalty on the mixing proportions. This algorithm was previously discussed, for instance, by Chen and Khalili (2008); Manole and Khalili (2021), and only differs from the traditional EM algorithm for Gaussian mixture models through the update on line 6. We used Algorithm 1 as written for Model B, whereas for Models A and C, we omitted the update on line 8 for the scale…”

Section: C1 Numerical Specificationsmentioning

confidence: 99%

See 2 more Smart Citations

Refined Convergence Rates for Maximum Likelihood Estimation under Finite Mixture Models

Manole¹,

Ho²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

We revisit convergence rates for maximum likelihood estimation (MLE) under finite mixture models. The Wasserstein distance has become a standard loss function for the analysis of parameter estimation in these models, due in part to its ability to circumvent label switching and to accurately characterize the behaviour of fitted mixture components with vanishing weights. However, the Wasserstein metric is only able to capture the worst-case convergence rate among the remaining fitted mixture components. We demonstrate that when the log-likelihood function is penalized to discourage vanishing mixing weights, stronger loss functions can be derived to resolve this shortcoming of the Wasserstein distance. These new loss functions accurately capture the heterogeneity in convergence rates of fitted mixture components, and we use them to sharpen existing pointwise and uniform convergence rates in various classes of mixture models. In particular, these results imply that a subset of the components of the penalized MLE typically converge significantly faster than could have been anticipated from past work. We further show that some of these conclusions extend to the traditional MLE. Our theoretical findings are supported by a simulation study to illustrate these improved convergence rates.

show abstract

Section: A Proofssupporting

confidence: 70%

Section: Convergence Rates For Maximum Likelihood Density Estimatorsmentioning

confidence: 99%

Section: C1 Numerical Specificationsmentioning

confidence: 99%

See 1 more Smart Citation

Refined Convergence Rates for Maximum Likelihood Estimation under Finite Mixture Models

Manole¹,

Ho²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are two popular line of directions to account for the unknown number of components. The first line of directions consist of approaches that penalize the sample log-likelihood function of the Gaussian mixture models via the number of parameters [25,24] or via the separation of the parameters [22]. While these methods can guarantee the consistency of estimating the true number of components when the sample size is sufficiently large, they tend to be computationally expensive, especially when the true number of components is quite large.…”

Section: Introductionmentioning

confidence: 99%

Beyond EM Algorithm on Over-specified Two-Component Location-Scale Gaussian Mixtures

Ren¹,

Cui²,

Sanghavi³

et al. 2022

Preprint

View full text Add to dashboard Cite

The Expectation-Maximization (EM) algorithm has been predominantly used to approximate the maximum likelihood estimation of the location-scale Gaussian mixtures. However, when the models are over-specified, namely, the chosen number of components to fit the data is larger than the unknown true number of components, EM needs a polynomial number of iterations in terms of the sample size to reach the final statistical radius; this is computationally expensive in practice. The slow convergence of EM is due to the missing of the locally strong convexity with respect to the location parameter on the negative population log-likelihood function, i.e., the limit of the negative sample log-likelihood function when the sample size goes to infinity. To efficiently explore the curvature of the negative log-likelihood functions, by specifically considering two-component location-scale Gaussian mixtures, we develop the Exponential Location Update (ELU) algorithm. The idea of the ELU algorithm is that we first obtain the exact optimal solution for the scale parameter and then perform an exponential step-size gradient descent for the location parameter. We demonstrate theoretically and empirically that the ELU iterates converge to the final statistical radius of the models after a logarithmic number of iterations. To the best of our knowledge, it resolves the long-standing open question in the literature about developing an optimization algorithm that has optimal statistical and computational complexities for solving parameter estimation even under some specific settings of the over-specified Gaussian mixture models.

show abstract

“…In the statistical literature, the clustering problem is commonly formulated as fitting a mixture model [13,18,24]. That is, suppose the observed random vector X is p-dimensional and is absolutely continuous with respect to the Lebesgue measure ζ p on R p ; we assume that f 0 (•), the probability density function (pdf) of X, has the form…”

Section: Introductionmentioning

confidence: 99%

Determine the number of clusters by data augmentation

Luo

2022

Electron. J. Statist.

View full text Add to dashboard Cite

Determining the number of clusters is crucial for the successful application of clustering. In this paper, we propose a new order-determination method called the data augmentation estimator (DAE), for the general model-based clustering. The estimator is based on a novel idea that augments data with an independently generated small cluster, which enables us to justify how the instability of clustering changes with the number of clusters assumed in clustering. The pattern of instability provides an alternative characterization of the true number of clusters to the commonly used goodness-of-fit measure. By combining the two sources of information appropriately, the proposed estimator reaches asymptotic consistency under general conditions and is easily implementable. It is also more efficient than the conventional BIC-type approaches that use the goodness-of-fit measure only. These properties are illustrated by the simulation studies and real data examples at the end.

show abstract

Estimating the number of components in finite mixture models via the Group-Sort-Fuse procedure

Cited by 12 publications

References 52 publications

Refined Convergence Rates for Maximum Likelihood Estimation under Finite Mixture Models

Refined Convergence Rates for Maximum Likelihood Estimation under Finite Mixture Models

Beyond EM Algorithm on Over-specified Two-Component Location-Scale Gaussian Mixtures

Determine the number of clusters by data augmentation

Contact Info

Product

Resources

About