Consistency of sparse PCA in High Dimension, Low Sample Size contexts

Abstract

Sparse Principal Component Analysis (PCA) methods are efficient tools to reduce the dimension (or the number of variables) of complex data. Sparse principal components (PCs) are easier to interpret than conventional PCs, because most loadings are zero. We study the asymptotic properties of these sparse PC directions for scenarios with fixed sample size and increasing dimension (i.e. High Dimension, Low Sample Size (HDLSS)). Under the previously studied spike covariance assumption, we show that Sparse PCA remains consistent under the same large spike condition that was previously established for conventional PCA. Under a broad range of small spike conditions, we find a large set of sparsity assumptions where Sparse PCA is consistent, but PCA is strongly inconsistent. The boundaries of the consistent region are clarified using an oracle result.

Projected Principal Component Analysis in Factor Models

“…In recent years, substantial work has been done on the PCA consistency on the spiked covariance model (e.g., Johnstone (2001) and Paul (2007)), and is extended to the HDLSS context by Ahn et al (2007), Jung and Marron (2009) and Shen et al (2013a). In a highdimensional factor model y t = Λf t + u t , let Σ = cov(y t ) be the p × p covariance matrix of y t .…”

Overview of object oriented data analysis

“…Most of this is asymptotics based, and OODA makes it clear that there are several important modes of asymptotics that can be considered. Even for Euclidean PCA, as made clear in Shen et al (2012a), several distinct asymptotic domains have been considered, ranging from the classical n → ∞, where d is fixed, through Random Matrix Theory where n ∼ d → ∞, see Johnstone (2006) and Johnstone and Lu (2009) for access to this literature, to High Dimension Low Sample Size (HDLSS) asymptotics where d → ∞ while n is fixed, see Jung and Marron (2009) for discussion of PCA in that context. In many OODA problems, the HDLSS type of asymptotics is informative and relevant, because sample sizes are frequently relatively small, and complexity of objects and their representations are frequently very large.…”

A Literature Survey on High-Dimensional Sparse Principal Component Analysis

“…The main questions needed to be answered in sparse PCA is whether there has an algorithm not only asymptotically consistent but also computationally efficient. Theoretical research from statistical guarantees view of sparse PCA includes consistency [2,8,14,38,41,50,53,55], minimax risk bounds for estimating eigenvectors [40,[42][43]45,61], optimal sparsity level detection [4,44,48,59] and principal subspaces estimation [5,[15][16]36,9,40,51,57] have been established under various statistical models. Because most of the methods based on spiked covariance model, so we firstly given an introduction about spiked variance model and then give a high dimensional sparse PCA theoretical analysis review from above several aspects.…”

“…Shen et al [41] established conditions for consistency of a sparse PCA method in [11] when p and n is fixed. Yuan [98] also derived the convergence rate of TPower methods.…”