Minimax bounds for sparse PCA with noisy high-dimensional data

Abstract

We study the problem of estimating the leading eigenvectors of a high-dimensional population covariance matrix based on independent Gaussian observations. We establish a lower bound on the minimax risk of estimators under the l2 loss, in the joint limit as dimension and sample size increase to infinity, under various models of sparsity for the population eigenvectors. The lower bound on the risk points to the existence of different regimes of sparsity of the eigenvectors. We also propose a new method for estimating the eigenvectors by a two-stage coordinate selection scheme.

Principal component analysis: a review and recent developments

“…Recent research (e.g. [18,19]) has examined how well underlying 'population' PCs are estimated by the sample PCs in the case where n p, and it is shown that in some circumstances there is little resemblance between sample and population PCs. However, the results are typically based on a model for the data which has a very small number of structured PCs, and very many noise dimensions, and which has some links with recent work in RPCA (see §3c).…”

Recent developments of constructing adjacency matrix in network analysis

“…Third, the most relevant result to statisticians is the Tracy-Widom law (Tracy and Widom, 1996) studied by Widom (1996, 2000), Johnstone (2001Johnstone ( , 2008, Birnbaum et al (2013), and many others. The main result is as follows; Suppose that X = (X ij ) p×n has entries which are iid N (0, 1).…”

“…Recently, Pillai and Yin (2012) showed that the largest eigenvalue of the correlation matrix also follows the same distribution. Recently, Birnbaum et al (2013) studied minimax bounds in the sparse spiked distribution, where the covariance matrix Σ is not an identity matrix but…”

“…Therefore, the white Wishart corresponds to the null hypothesis (no hubs), and the spiked distribution corresponds to the alternative hypothesis (M hubs). See references in Birnbaum et al (2013) for studies in the spiked distribution.…”

“…Among them, Whittaker (1990), Edward (2000), Meinshausen and Buhlmann (2006), Yuan and Lin (2007), Peng et al (2009), Rothman et al (2010), Bien and Tibshirani (2011), Cai and Yuan (2012) and Chandrasekaran et al (2012) are often referred. Also, noticeable results on the distribution of the largest eigenvalue of the sample covariance matrix under various assumptions are done by Johnstone (2001Johnstone ( , 2008, Bickel and Levina (2008), Rothman et al (2009), Cai and Liu (2011), Cai and Zhou (2012 and Birnbaum et al (2013). Recently, Won and Choi (2014) applied the network theory to the comparison of citations in journals.…”