Minimax bounds for sparse PCA with noisy high-dimensional data

Birnbaum, Aharon; Johnstone, Iain M.; Nadler, Boaz; Paul, Debashis

doi:10.48550/arxiv.1203.0967

Cited by 3 publications

(16 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore the rate under the Frobenius norm far exceeds (9) when r ≫ log ep k . When r = 1, both norms lead to the same rate and the result in (9) recovers earlier results on estimating the leading eigenvector obtained in [5,39,28].…”

Section: Main Contributionssupporting

confidence: 86%

“…Proof of Theorem 4 1 • The minimax lower bound for estimating span(V) follows straightforwardly from previous results on estimating the leading singular vector, i.e., the rank-one case (see, e.g., [5,39]). The desired lower bound (25) can be found in [12,Eq.…”

Section: Proofs Of the Lower Boundsmentioning

confidence: 87%

“…See, for instance, [16,24,32,34]. In the high-dimensional setting, various aspects of this model have been studied by several recent papers, including but not limited to [1,5,12,20,21,31,33,35]. For simplicity, we assume σ is known.…”

Section: Sparse Spiked Covariance Matrix Modelmentioning

confidence: 99%

“…In many statistical applications, instead of the covariance matrix itself, the object of direct interest is often a lower dimensional functional of the covariance matrix, e.g., the principal subspace span(V). This problem is known in the literature as sparse PCA [5,12,20,31]. The third goal of the paper is the minimax estimation of the principal subspace span(V).…”

Section: Main Contributionsmentioning

confidence: 99%

“…where (75) follows from rank( v v ′ − vv ′ ) ≤ 2 and (76) follows from the minimax lower bound in [39, Theorem 2.1] (see also [5,Theorem 2]) for estimating the leading singular vector. 2.2 • To prove the lower bound λ 2 1 ∧ r n , consider the following (composite) hypotheses testing problem:…”

Section: Proofs Of the Lower Boundsmentioning

confidence: 99%

See 4 more Smart Citations

Optimal estimation and rank detection for sparse spiked covariance matrices

Cai

2014

Probab. Theory Relat. Fields

132

138

View full text Add to dashboard Cite

This paper considers a sparse spiked covariancematrix model in the high-dimensional setting and studies the minimax estimation of the covariance matrix and the principal subspace as well as the minimax rank detection. The optimal rate of convergence for estimating the spiked covariance matrix under the spectral norm is established, which requires significantly different techniques from those for estimating other structured covariance matrices such as bandable or sparse covariance matrices. We also establish the minimax rate under the spectral norm for estimating the principal subspace, the primary object of interest in principal component analysis. In addition, the optimal rate for the rank detection boundary is obtained. This result also resolves the gap in a recent paper by Berthet and Rigollet [2] where the special case of rank one is considered.

show abstract

Section: Main Contributionssupporting

confidence: 86%

Section: Proofs Of the Lower Boundsmentioning

confidence: 87%

Section: Sparse Spiked Covariance Matrix Modelmentioning

confidence: 99%

Section: Main Contributionsmentioning

confidence: 99%

Section: Proofs Of the Lower Boundsmentioning

confidence: 99%

See 3 more Smart Citations

Optimal estimation and rank detection for sparse spiked covariance matrices

Cai

2014

Probab. Theory Relat. Fields

132

138

View full text Add to dashboard Cite

show abstract

OptShrink: An Algorithm for Improved Low-Rank Signal Matrix Denoising by Optimal, Data-Driven Singular Value Shrinkage

Nadakuditi

2014

IEEE Trans. Inform. Theory

143

160

View full text Add to dashboard Cite

Abstract. The truncated singular value decomposition (SVD) of the measurement matrix is the optimal solution to the representation problem of how to best approximate a noisy measurement matrix using a low-rank matrix. Here, we consider the (unobservable) denoising problem of how to best approximate a low-rank signal matrix buried in noise by optimal (re)weighting of the singular vectors of the measurement matrix. We exploit recent results from random matrix theory to exactly characterize the large matrix limit of the optimal weighting coefficients and show that they can be computed directly from data for a large class of noise models that includes the i.i.d. Gaussian noise case.Our analysis brings into sharp focus the shrinkage-and-thresholding form of the optimal weights, the non-convex nature of the associated shrinkage function (on the singular values) and explains why matrix regularization via singular value thresholding with convex penalty functions (such as the nuclear norm) will always be suboptimal. We validate our theoretical predictions with numerical simulations, develop an implementable algorithm (OptShrink) that realizes the predicted performance gains and show how our methods can be used to improve estimation in the setting where the measured matrix has missing entries.

show abstract

Sparse PCA: Optimal rates and adaptive estimation

Cai

Wu³

2013

Ann. Statist.

234

281

View full text Add to dashboard Cite

Principal component analysis (PCA) is one of the most commonly used statistical procedures with a wide range of applications. This paper considers both minimax and adaptive estimation of the principal subspace in the high dimensional setting. Under mild technical conditions, we first establish the optimal rates of convergence for estimating the principal subspace which are sharp with respect to all the parameters, thus providing a complete characterization of the difficulty of the estimation problem in term of the convergence rate. The lower bound is obtained by calculating the local metric entropy and an application of Fano's lemma. The rate optimal estimator is constructed using aggregation, which, however, might not be computationally feasible. We then introduce an adaptive procedure for estimating the principal subspace which is fully data driven and can be computed efficiently. It is shown that the estimator attains the optimal rates of convergence simultaneously over a large collection of the parameter spaces. A key idea in our construction is a reduction scheme which reduces the sparse PCA problem to a high-dimensional multivariate regression problem. This method is potentially also useful for other related problems.Comment: Published in at http://dx.doi.org/10.1214/13-AOS1178 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

show abstract

Minimax bounds for sparse PCA with noisy high-dimensional data

Cited by 3 publications

References 0 publications

Optimal estimation and rank detection for sparse spiked covariance matrices

Optimal estimation and rank detection for sparse spiked covariance matrices

OptShrink: An Algorithm for Improved Low-Rank Signal Matrix Denoising by Optimal, Data-Driven Singular Value Shrinkage

Sparse PCA: Optimal rates and adaptive estimation

Contact Info

Product

Resources

About