Polynomial-Time Tensor Decompositions with Sum-of-Squares

Ma, Tengyu; Shi, Jonathan; Steurer, David

doi:10.1109/focs.2016.54

Cited by 62 publications

(104 citation statements)

References 27 publications

(60 reference statements)

Supporting

Mentioning

104

Contrasting

Order By: Relevance

“…Initiated by the work of [BBH + 12], who showed that polynomial time algorithms in the hierarchy solve all known integrality gap instances for Unique Games and related problems, a steady stream of works have developed efficient algorithms for both worst-case [BKS14, BKS15, BKS17, BGG + 16] and average-case problems [HSS15,GM15,BM16,RRS16,BGL16,MSS16a,PS17]. The insights from these works extend beyond individual algorithms to characterizations of broad classes of algorithmic techniques.…”

Section: Introductionmentioning

confidence: 99%

The Power of Sum-of-Squares for Detecting Hidden Structures

Hopkins

Kothari²,

Potechin³

et al. 2017

2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)

Self Cite

View full text Add to dashboard Cite

We study planted problems-finding hidden structures in random noisy inputs-through the lens of the sum-of-squares semidefinite programming hierarchy (SoS). This family of powerful semidefinite programs has recently yielded many new algorithms for planted problems, often achieving the best known polynomial-time guarantees in terms of accuracy of recovered solutions and robustness to noise. One theme in recent work is the design of spectral algorithms which match the guarantees of SoS algorithms for planted problems. Classical spectral algorithms are often unable to accomplish this: the twist in these new spectral algorithms is the use of spectral structure of matrices whose entries are low-degree polynomials of the input variables.We prove that for a wide class of planted problems, including refuting random constraint satisfaction problems, tensor and sparse PCA, densest-k-subgraph, community detection in stochastic block models, planted clique, and others, eigenvalues of degree-d matrix polynomials are as powerful as SoS semidefinite programs of size roughly n d . For such problems it is therefore always possible to match the guarantees of SoS without solving a large semidefinite program.Using related ideas on SoS algorithms and low-degree matrix polynomials (and inspired by recent work on SoS and the planted clique problem [BHK + 16]), we prove new nearly-tight SoS lower bounds for the tensor and sparse principal component analysis problems. Our lower bounds are the first to suggest that improving upon the signal-to-noise ratios handled by existing polynomial-time algorithms for these problems may require subexponential time.

show abstract

Section: Introductionmentioning

confidence: 99%

The Power of Sum-of-Squares for Detecting Hidden Structures

Hopkins

Kothari²,

Potechin³

et al. 2017

2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The remarkable result of Kruskal [Kru77] shows that for s > 2, the decomposition in "typically" unique, as long as R is a small enough. Several works [Har70, Car91, AGH + 12, MSS16] have designed efficient recovery algorithms in different regimes of R, and assumptions on {A i }. The other important question is if the {A i } can be recovered assuming that we only have access to M s + Err, for some noise tensor Err.…”

Section: Overcomplete Tensor Decompositionsmentioning

confidence: 99%

“…Similarly, parameter estimation for basic latent variable models like mixtures of spherical Gaussians has exponential sample complexity in the worst case [MV10]; yet, polynomial time guarantees can be obtained using smoothed analysis, where the parameters (e.g., means for Gaussians) are randomly perturbed in high dimensions [HK12, BCMV14, ABG + 14, GHK15]. 1 Smoothed analysis results have also been obtained for other problems like overcomplete ICA [GVX14], learning mixtures of general Gaussians [GHK15], fourth-order tensor decompositions [MSS16], and recovering assemblies of neurons [ADM + 18].…”

Section: Introductionmentioning

confidence: 99%

Smoothed Analysis in Unsupervised Learning via Decoupling

Bhaskara

Chen

Perreault

et al. 2019

2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS)

View full text Add to dashboard Cite

Smoothed analysis is a powerful paradigm in overcoming worst-case intractability in unsupervised learning and high-dimensional data analysis. While polynomial time smoothed analysis guarantees have been obtained for worst-case intractable problems like tensor decompositions and learning mixtures of Gaussians, such guarantees have been hard to obtain for several other important problems in unsupervised learning. A core technical challenge in analyzing algorithms is obtaining lower bounds on the least singular value for random matrix ensembles with dependent entries, that are given by low-degree polynomials of a few base underlying random variables.In this work, we address this challenge by obtaining high-confidence lower bounds on the least singular value of new classes of structured random matrix ensembles of the above kind. We then use these bounds to design algorithms with polynomial time smoothed analysis guarantees for the following three important problems in unsupervised learning:• Robust subspace recovery, when the fraction α of inliers in the d-dimensional subspace T ⊂ R n is at least α > (d/n) ℓ for any constant ℓ ∈ Z + . This contrasts with the known worst-case intractability when α < d/n, and the previous smoothed analysis result which needed α > d/n (Hardt and Moitra, 2013).• Learning overcomplete hidden markov models, where the size of the state space is any polynomial in the dimension of the observations. This gives the first polynomial time guarantees for learning overcomplete HMMs in the smoothed analysis model.• Higher order tensor decompositions, where we generalize and analyze the so-called FOOBI algorithm of Cardoso to find order-ℓ rank-one tensors in a subspace. This allows us to obtain polynomially robust decomposition algorithms for 2ℓ'th order tensors with rank O(n ℓ ).

show abstract

“…where the a i ∈ R p are drawn independently from N (0, I/p). The state-of-the-art results for this problem are a close-to-linear-time spectral method that succeeds when r p 4/3 [HSSS16] and a polynomial-time sum-of-squares method that succeeds when r p 3/2 [MSS16]. (It seems likely that no efficient algorithm can succeed when r exceeds p 3/2 .…”

Section: Summary Of Techniquesmentioning

confidence: 99%

“…In the overcomplete case, a line of work has culminated in a polynomial time algorithm that works when the vectors a i are random (i.i.d. Gaussian) and r p 3/2 [GM15, HSSS16,MSS16]. In applications, the vectors a 1 , .…”

Section: Introductionmentioning

confidence: 99%

Spectral methods from tensor networks

Moitra

Wein

2019

Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing

View full text Add to dashboard Cite

A tensor network is a diagram that specifies a way to "multiply" a collection of tensors together to produce another tensor (or matrix). Many existing algorithms for tensor problems (such as tensor decomposition and tensor PCA), although they are not presented this way, can be viewed as spectral methods on matrices built from simple tensor networks. In this work we leverage the full power of this abstraction to design new algorithms for certain continuous tensor decomposition problems.An important and challenging family of tensor problems comes from orbit recovery, a class of inference problems involving group actions (inspired by applications such as cryo-electron microscopy). Orbit recovery problems over finite groups can often be solved via standard tensor methods. However, for infinite groups, no general algorithms are known. We give a new spectral algorithm based on tensor networks for one such problem: continuous multi-reference alignment over the infinite group SO(2). Our algorithm extends to the more general heterogeneous case. *

show abstract

Polynomial-Time Tensor Decompositions with Sum-of-Squares

Cited by 62 publications

References 27 publications

The Power of Sum-of-Squares for Detecting Hidden Structures

The Power of Sum-of-Squares for Detecting Hidden Structures

Smoothed Analysis in Unsupervised Learning via Decoupling

Spectral methods from tensor networks

Contact Info

Product

Resources

About