Sharp optimal recovery in the two-component Gaussian Mixture Model

Ndaoud, Mohamed

doi:10.48550/arxiv.1812.08078

Cited by 16 publications

(37 citation statements)

References 12 publications

(32 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When Σ is known and the noise is normal, is it easy to see that Σ −1/2 Y follows the isotropic Gaussian Mixture Model where the signal vector is given by Σ −1/2 θ. Following the reasoning similar to [12], we can show in the general case that inf…”

Section: Statement Of the Problemmentioning

confidence: 83%

“…This particular choice is motivated by the fact that we are seeking bounds that hold adaptively for large classes of Σ which is typically unknown in practical applications. The proof of the lower bound is inspired by the argument in [12] that only holds for isotropic noise. As for the upper bound, we show that the "averaging" linear classifier defined as ηave (y) := sign n i=1 Y i η i , y , is minimax optimal.…”

Section: Minimax Clustering: the Supervised Casementioning

confidence: 99%

See 1 more Smart Citation

Minimax Supervised Clustering in the Anisotropic Gaussian Mixture Model: A new take on Robust Interpolation

Minsker¹,

Ndaoud²,

Shen³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We study the supervised clustering problem under the twocomponent anisotropic Gaussian mixture model in high dimensions and in the non-asymptotic setting. We first derive a lower and a matching upper bound for the minimax risk of clustering in this framework. We also show that in the high-dimensional regime, the linear discriminant analysis (LDA) classifier turns out to be sub-optimal in the minimax sense. Next, we characterize precisely the risk of 2 -regularized supervised least squares classifiers. We deduce the fact that the interpolating solution may outperform the regularized classifier, under mild assumptions on the covariance structure of the noise. Our analysis also shows that interpolation can be robust to corruption in the covariance of the noise when the signal is aligned with the "clean" part of the covariance, for the properly defined notion of alignment. To the best of our knowledge, this peculiar phenomenon has not yet been investigated in the rapidly growing literature related to interpolation. We conclude that interpolation is not only benign but can also be optimal, and in some cases robust.

show abstract

Section: Statement Of the Problemmentioning

confidence: 83%

Section: Minimax Clustering: the Supervised Casementioning

confidence: 99%

Minimax Supervised Clustering in the Anisotropic Gaussian Mixture Model: A new take on Robust Interpolation

Minsker¹,

Ndaoud²,

Shen³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…, ĜK . On the other hand, it is known that rounding is not necessary as the relaxed SDP solution can be directly used to recover the K-means in (1), when the separation of cluster centers is large enough, a property often referred in literature as the hidden integrality [13,10,25,6].…”

Section: Sdp Relaxed K-meansmentioning

confidence: 99%

Sketch-and-Lift: Scalable Subsampled Semidefinite Program for $K$-means Clustering

Zhuang¹,

Chen²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

Semidefinite programming (SDP) is a powerful tool for tackling a wide range of computationally hard problems such as clustering. Despite the high accuracy, semidefinite programs are often too slow in practice with poor scalability on large (or even moderate) datasets. In this paper, we introduce a linear time complexity algorithm for approximating an SDP relaxed K-means clustering. The proposed sketch-and-lift (SL) approach solves an SDP on a subsampled dataset and then propagates the solution to all data points by a nearest-centroid rounding procedure. It is shown that the SL approach enjoys a similar exact recovery threshold as the K-means SDP on the full dataset, which is known to be information-theoretically tight under the Gaussian mixture model. The SL method can be made adaptive with enhanced theoretic properties when the cluster sizes are unbalanced. Our simulation experiments demonstrate that the statistical accuracy of the proposed method outperforms state-of-the-art fast clustering algorithms without sacrificing too much computational efficiency, and is comparable to the original K-means SDP with substantially reduced runtime.

show abstract

“…This framework is indeed a good benchmark, since on the one hand, it is sufficiently simple to allow us to understand the nature of the target (β 0,1 , ..., β 0,K ) -with K = 2 and β 0,1 = −β 0,2 in our bipartite framework -and to investigate the rate of convergence of estimators of the form of (2), suitably regularized by a 1 -penalty. On the other hand, the two-component high-dimensional Gaussian mixture has received recently at lot of attention [7,2,35,28,24,15,12,3,25,8,30]. Let us emphasize that our goal is not a priori to provide a state-of-the-art method, specifically designed to solve the high-dimensional…”

Section: Introductionmentioning

confidence: 99%

High-dimensional logistic entropy clustering

Genetay¹,

Saumard²,

Coulaud³

2021

Preprint

View full text Add to dashboard Cite

Minimization of the (regularized) entropy of classification probabilities is a versatile class of discriminative clustering methods. The classification probabilities are usually defined through the use of some classical losses from supervised classification and the point is to avoid modelisation of the full data distribution by just optimizing the law of the labels conditioned on the observations. We give the first theoretical study of such methods, by specializing to logistic classification probabilities. We prove that if the observations are generated from a two-component isotropic Gaussian mixture, then minimizing the entropy risk over a Euclidean ball indeed allows to identify the separation vector of the mixture. Furthermore, if this separation vector is sparse, then penalizing the empirical risk by a 1 -regularization term allows to infer the separation in a high-dimensional space and to recover its support, at standard rates of sparsity problems. Our approach is based on the local convexity of the logistic entropy risk, that occurs if the separation vector is large enough, with a condition on its norm that is independent from the space dimension. This local convexity property also guarantees fast rates in a classical, low-dimensional setting.

show abstract

Sharp optimal recovery in the two-component Gaussian Mixture Model

Cited by 16 publications

References 12 publications

Minimax Supervised Clustering in the Anisotropic Gaussian Mixture Model: A new take on Robust Interpolation

Minimax Supervised Clustering in the Anisotropic Gaussian Mixture Model: A new take on Robust Interpolation

Sketch-and-Lift: Scalable Subsampled Semidefinite Program for $K$-means Clustering

High-dimensional logistic entropy clustering

Contact Info

Product

Resources

About