Spectral Methods for Data Science: A Statistical Perspective

Chen, Yuxin; Chi, Yuejie; Fan, Jianqing; Ma, Cong

doi:10.1561/2200000079

Cited by 70 publications

(52 citation statements)

References 255 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lemma D.7 (Wedin's sin Θ Theorem, Theorem 2.3.1 in [13]). Let M and M = M + E be two n 1 × n 2 real matrices and n 1 ≥ n 2 , with SVDs given respectively by…”

Section: K} Then With Probability At Leastmentioning

confidence: 99%

Partial recovery and weak consistency in the non-uniform hypergraph Stochastic Block Model

Dumitriu¹,

Wang²,

Zhu³

2021

Preprint

View full text Add to dashboard Cite

We consider the community detection problem in sparse random hypergraphs under the nonuniform hypergraph stochastic block model (HSBM), a general model of random networks with community structure and higher-order interactions. When the random hypergraph has bounded expected degrees, we provide a spectral algorithm that outputs a partition with at least a γ fraction of the vertices classified correctly, where γ ∈ (0.5, 1) depends on the signal-to-noise ratio (SNR) of the model. When the SNR grows slowly as the number of vertices goes to infinity, our algorithm achieves weak consistency, which improves the previous results in [24] for non-uniform HSBMs.Our spectral algorithm consists of three major steps: (1) Hyperedge selection: select hyperedges of certain sizes to provide the maximal signal-to-noise ratio for the induced sub-hypergraph; (2) Spectral partition: construct a regularized adjacency matrix and obtain an approximate partition based on singular vectors;(3) Correction and merging: incorporate the hyperedge information from adjacency tensors to upgrade the error rate guarantee. The theoretical analysis of our algorithm relies on the concentration and regularization of the adjacency matrix for sparse non-uniform random hypergraphs, which can be of independent interest.

show abstract

“…Lemma D.7 (Wedin's sin Θ Theorem, Theorem 2.3.1 in [13]). Let M and M = M + E be two n 1 × n 2 real matrices and n 1 ≥ n 2 , with SVDs given respectively by…”

Section: K} Then With Probability At Leastmentioning

confidence: 99%

Partial recovery and weak consistency in the non-uniform hypergraph Stochastic Block Model

Dumitriu¹,

Wang²,

Zhu³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…G k−1 , plug them into (2) to solve for G k . In principle, this is the same as solving (3) where each G 1 , . .…”

Section: Main Idea Of the Algorithmmentioning

confidence: 99%

“…Now that B k is obtained as the left singular vectors of Φ k in Trimming, we invoke Wedin's theorem [21] to quantify ∆B k in terms of ∆ Φ k . To this end, we first introduce the following distance comparing two 3-tensors up to rotation, which is common in spectral analysis of linear algebra, see Chapter 2 of [3].…”

mentioning

confidence: 99%

“…Recall that B 1 and B1 are the first r 1 left singular vectors of p(x 1 ; x 2 ) and p(x 1 ; x 2 ), respectively. We apply Wedin's theorem presented in Theorem 2.9 of [3]; if ∆p(x 1 ; x 2 ) < σ r1 (p(x 1 ; x 2 )), we can find R 1 ∈ O(r 1 ) such that B1 (x 1 ; α 1 ) = r1 a1=1 B 1 (x 1 ; a 1 )R 1 (a 1 ; α 1 ) + E 1 (x 1 ; α 1 ), and E 1 (x 1 ; α 1 ) ≤ √ 2 ∆p(x 1 ; x 2 ) B 1 (x 1 ; a 1 ) σ r1 (p(x 1 ; x 2 )) − ∆p(x 1 ; x 2 ) .…”

mentioning

confidence: 99%

See 1 more Smart Citation

Generative modeling via tensor train sketching

Hur¹,

Hoskins²,

Lindsey³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper we introduce a sketching algorithm for constructing a tensor train representation of a probability density from its samples. Our method deviates from the standard recursive SVD-based procedure for constructing a tensor train. Instead we formulate and solve a sequence of small linear systems for the individual tensor train cores. This approach can avoid the curse of dimensionality that threatens both the algorithmic and sample complexities of the recovery problem. Specifically, for Markov models, we prove that the tensor cores can be recovered with a sample complexity that is constant with respect to the dimension. Finally, we illustrate the performance of the method with several numerical experiments.

show abstract

“…Algorithm 1 describes the gradient descent algorithm initialized by the spectral method [12] for minimizing (8). Compared to the Procrustes Flow (PF) algorithm in [7], which minimizes the regularized loss function in (5), the new algorithm does not include the balancing regularizer g(X, Y ).…”

Section: Low-rank Matrix Sensingmentioning

confidence: 99%

Beyond Procrustes: Balancing-Free Gradient Descent for Asymmetric Low-Rank Matrix Sensing

Ma,

Li,

Chi

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Low-rank matrix estimation plays a central role in various applications across science and engineering. Recently, nonconvex formulations based on matrix factorization are provably solved by simple gradient descent algorithms with strong computational and statistical guarantees. However, when the low-rank matrices are asymmetric, existing approaches rely on adding a regularization term to balance the scale of the two matrix factors which in practice can be removed safely without hurting the performance when initialized via the spectral method. In this paper, we provide a theoretical justification to this for the matrix sensing problem, which aims to recover a low-rank matrix from a small number of linear measurements. As long as the measurement ensemble satisfies the restricted isometry property, gradient descent-in conjunction with spectral initialization-converges linearly without the need of explicitly promoting balancedness of the factors; in fact, the factors stay balanced automatically throughout the execution of the algorithm. Our analysis is based on analyzing the evolution of a new distance metric that directly accounts for the ambiguity due to invertible transforms, and might be of independent interest.

show abstract

Spectral Methods for Data Science: A Statistical Perspective

Cited by 70 publications

References 255 publications

Partial recovery and weak consistency in the non-uniform hypergraph Stochastic Block Model

Partial recovery and weak consistency in the non-uniform hypergraph Stochastic Block Model

Generative modeling via tensor train sketching

Beyond Procrustes: Balancing-Free Gradient Descent for Asymmetric Low-Rank Matrix Sensing

Contact Info

Product

Resources

About