Classification of sparse high-dimensional vectors

Ingster, Yuri I.; Pouet, Christophe; Tsybakov, Alexandre B.

doi:10.1098/rsta.2009.0156

Cited by 31 publications

(52 citation statements)

References 14 publications

(20 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At the same time, by a direct use of the elementary Random Matrix Theory [45], H 0 = ZZ − pI p ≤ C √ pn. Combining these with (C.25)-(C. 26) gives (C.27)…”

Section: Appendix B: Proof Of Lemmas In Sectionmentioning

confidence: 97%

Phase transitions for high dimensional clustering and related problems

Jin¹,

Ke²,

Wang³

2017

Ann. Statist.

View full text Add to dashboard Cite

Consider a two-class clustering problem where we observe Xi = iµ + Zi, Zi iid ∼ N (0, Ip), 1 ≤ i ≤ n. The feature vector µ ∈ R p is unknown but is presumably sparse. The class labels i ∈ {−1, 1} are also unknown and the main interest is to estimate them.We are interested in the statistical limits. In the two-dimensional phase space calibrating the rarity and strengths of useful features, we find the precise demarcation for the Region of Impossibility and Region of Possibility. In the former, useful features are too rare/weak for successful clustering. In the latter, useful features are strong enough to allow successful clustering. The results are extended to the case of colored noise using Le Cam's idea on comparison of experiments.We also extend the study on statistical limits for clustering to that for signal recovery and that for global testing. We compare the statistical limits for three problems and expose some interesting insight.We propose classical PCA and Important Features PCA (IF-PCA) for clustering. For a threshold t > 0, IF-PCA clusters by applying classical PCA to all columns of X with an L 2 -norm larger than t. We also propose two aggregation methods. For any parameter in the Region of Possibility, some of these methods yield successful clustering.We discover a phase transition for IF-PCA. For any threshold t > 0, let ξ (t) be the first left singular vector of the post-selection data matrix. The phase space partitions into two different regions. In one region, there is a t such that cos(ξ (t) , ) → 1 and IF-PCA yields successful clustering. In the other, cos(ξ (t) , ) ≤ c0 < 1 for all t > 0. Our results require delicate analysis, especially on post-selection Random Matrix Theory and on lower bound arguments.

show abstract

“…At the same time, by a direct use of the elementary Random Matrix Theory [45], H 0 = ZZ − pI p ≤ C √ pn. Combining these with (C.25)-(C. 26) gives (C.27)…”

Section: Appendix B: Proof Of Lemmas In Sectionmentioning

confidence: 97%

Phase transitions for high dimensional clustering and related problems

Jin¹,

Ke²,

Wang³

2017

Ann. Statist.

View full text Add to dashboard Cite

show abstract

“…for some absolute constant C * > 0, where the last inequality follows from (17). Choosing A ε as a solution of C * A −2 ε = ε we obtain (22). The case 0 < q < 2 is treated analogously by introducing the test…”

Section: Consequences For the Problem Of Testingmentioning

confidence: 99%

Minimax estimation of linear and quadratic functionals on sparsity classes

Collier

Comminges²,

Tsybakov³

2017

Ann. Statist.

Self Cite

View full text Add to dashboard Cite

For the Gaussian sequence model, we obtain non-asymp-totic minimax rates of estimation of the linear, quadratic and the ℓ 2 -norm functionals on classes of sparse vectors and construct optimal estimators that attain these rates. The main object of interest is the class B 0 (s) of s-sparse vectors θ = (θ 1 , . . . , θ d ), for which we also provide completely adaptive estimators (independent of s and of the noise variance σ) having only logarithmically slower rates than the minimax ones. Furthermore, we obtain the minimax rates on. This analysis shows that there are, in general, three zones in the rates of convergence that we call the sparse zone, the dense zone and the degenerate zone, while a fourth zone appears for estimation of the quadratic functional. We show that, as opposed to estimation of θ, the correct logarithmic terms in the optimal rates for the sparse zone scale as log(d/s 2 ) and not as log(d/s). For the class B 0 (s), the rates of estimation of the linear functional and of the ℓ 2 -norm have a simple elbow at s = √ d (boundary between the sparse and the dense zones) and exhibit similar performances, whereas the estimation of the quadratic functional Q(θ) reveals more complex effects and is not possible only on the basis of sparsity described by the condition θ ∈ B 0 (s). Finally, we apply our results on estimation of the ℓ 2 -norm to the problem of testing against sparse alternatives. In particular, we obtain a non-asymptotic analog of Ingster-Donoho-Jin theory revealing some effects that were not captured by the previous asymptotic analysis.

show abstract

“…In particular, note that ρ l ρ k = ρ l ρ k (1 − γ 2 p 1 ,N ). Repeating the proof of Theorem 1 but with ρ l and ρ k and under the stronger condition (33), obtain P (l = l |x = x) ≤ 2α that, together with (18) and P (x = x) ≤ 2α, completes the proof.…”

Section: Proof Of Theoremmentioning

confidence: 61%

Classification with many classes: Challenges and pluses

Abramovich

Pensky

2019

Journal of Multivariate Analysis

View full text Add to dashboard Cite

The objective of the paper is to study accuracy of multi-class classification in highdimensional setting, where the number of classes is also large ("large L, large p, small n" model). While this problem arises in many practical applications and many techniques have been recently developed for its solution, to the best of our knowledge nobody provided a rigorous theoretical analysis of this important setup. The purpose of the present paper is to fill in this gap.We consider one of the most common settings, classification of high-dimensional normal vectors where, unlike standard assumptions, the number of classes could be large. We derive non-asymptotic conditions on effects of significant features, and the low and the upper bounds for distances between classes required for successful feature selection and classification with a given accuracy. Furthermore, we study an asymptotic setup where the number of classes is growing with the dimension of feature space and while the number of samples per class is possibly limited. We discover an interesting and, at first glance, somewhat counter-intuitive phenomenon that a large number of classes may be a "blessing" rather than a "curse" since, in certain settings, the precision of classification can improve as the number of classes grows. This is due to more accurate feature selection since even weaker significant features, which are not sufficiently strong to be manifested in a coarse classification, can nevertheless have a strong impact when the number of classes is large. We supplement our theoretical investigation by a simulation study and a real data example where we again observe the above phenomenon.

show abstract

Classification of sparse high-dimensional vectors

Cited by 31 publications

References 14 publications

Phase transitions for high dimensional clustering and related problems

Phase transitions for high dimensional clustering and related problems

Minimax estimation of linear and quadratic functionals on sparsity classes

Classification with many classes: Challenges and pluses

Contact Info

Product

Resources

About