Optimal Average-Case Reductions to Sparse PCA: From Weak Assumptions to Strong Hardness

Abstract

In the past decade, sparse principal component analysis has emerged as an archetypal problem for illustrating statistical-computational tradeoffs. This trend has largely been driven by a line of research aiming to characterize the average-case complexity of sparse PCA through reductions from the planted clique (PC) conjecture -which conjectures that there is no polynomial-time algorithm to detect a planted clique of sizeAll of these reductions either fail to show tight computational lower bounds matching existing algorithms or show lower bounds for formulations of sparse PCA other than its canonical generative model, the spiked covariance model. Also, these lower bounds all quickly degrade with the exponent in the PC conjecture. Specifically, when only given the PC conjecture up to K = o(N α ) where α < 1/2, there is no sparsity level k at which these lower bounds remain tight. If α ≤ 1/3 these reductions fail to even show the existence of a statistical-computational tradeoff at any sparsity k. Our main results are as follows:• We give a reduction from PC that yields the first full characterization of the computational barrier in the spiked covariance model, providing tight lower bounds at all sparsities k. This partially resolves a question raised in [BBH18].• We show the surprising result that weaker forms of the PC conjecture up to clique size K = o(N α ) for any given α ∈ (0, 1/2] imply tight computational lower bounds for sparse PCA at sparsities k = o(n α/3 ). This shows that even a mild improvement in the signal strength needed by the best known polynomial-time sparse PCA algorithms would imply that the hardness threshold for PC is subpolynomial, rather than on the order N 1/2 as is widely conjectured.

Universality of Computational Lower Bounds for Submatrix Detection

“…This work is part of a growing body of literature establishing statistical-computational gaps in high-dimensional inference problems based on average-case reductions. Previous reductions include lower bounds for testing k-wise independence [AAK + 07], RIP certification [WBP16,KZ14], matrix completion [Che15] and sparse PCA [BR13b,BR13a,WBS16,GMZ17,BB19]. A number of techniques were introduced in [BBH18] to provide the first web of average-case reductions to problems including planted independent set, planted dense subgraph, sparse spiked Wigner, sparse PCA, the subgraph stochastic block model and biclustering.…”

Adversarially Robust Low Dimensional Representations

“…The work of [GMZ + 17b] was the first to provide computational lower bounds for sparse PCA under the spiked covariance model. These bounds have been further improved in the work of [BBH18,BB19a]. The work of [WLL14] studies computationally efficient estimation of subspaces under the spiked covariance and more general models.…”

Average-Case Lower Bounds for Learning Sparse Mixtures, Robust Estimation and Semirandom Adversaries

“…We now state the total variation guarantees of Gaussianize. The instantiation of Gaussianize here generalizes that in [BB19] to rectangular matrices, but has the same proof. The procedure applies a Gaussian rejection kernel entrywise and its total variation guarantees follow by a simple by applying the tensorization property of d TV from Fact 3.2.…”

“…We also will require the subroutine Gaussianize from [BB19], shown in Figure 4, which maps a planted Bernoulli submatrix problem to a corresponding submatrix problems with independent Gaussian entries. To describe this subroutine, we first will need the univariate rejection kernel framework introduced in [BBH18].…”

“…The procedure applies a Gaussian rejection kernel entrywise and its total variation guarantees follow by a simple by applying the tensorization property of d TV from Fact 3.2. Lemma 4.5 (Gaussianization -Lemma 4.5 in [BB19]). Given parameters m and n, let 0 < Q < P ≤ 1 be such that…”