Optimal link prediction with matrix logistic regression

Baldin, Nicolai; Berthet, Quentin

doi:10.48550/arxiv.1803.07054

Cited by 6 publications

(8 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work is also related to a line of works on average-case computational hardness and the statistical and computational trade-offs. The average-case reduction approach has been commonly used to show computational lower bounds for many recent high-dimensional problems, such as testing k-wise independence (Alon et al, 2007), biclustering (Ma and Wu, 2015;Cai et al, 2017;Cai and Wu, 2018), community detection (Hajek et al, 2015), RIP certification (Wang et al, 2016a;Koiran and Zouzias, 2014), matrix completion (Chen, 2015), sparse PCA (Berthet and Rigollet, 2013a,b;Brennan et al, 2018;Gao et al, 2017;Wang et al, 2016b), universal submatrix detection , sparse mixture and robust estimation , a financial model with asymmetry information (Arora et al, 2011), finding dense common subgraphs (Charikar et al, 2018), link prediction (Baldin and Berthet, 2018), online local learning (Awasthi et al, 2015). See also a web of average-case reduction to a number of problems in Brennan et al (2018).…”

Section: Related Literaturementioning

confidence: 99%

Tensor Clustering with Planted Structures: Statistical Optimality and Computational Limits

Luo¹,

Zhang²

2020

Preprint

View full text Add to dashboard Cite

This paper studies the statistical and computational limits of high-order clustering with planted structures. We focus on two clustering models, constant high-order clustering (CHC) and rank-one higher-order clustering (ROHC), and study the methods and theories for testing whether a cluster exists (detection) and identifying the support of cluster (recovery).Specifically, we identify sharp boundaries of signal-to-noise ratio for which CHC and ROHC detection/recovery are statistically possible. We also develop tight computational thresholds: when the signal-to-noise ratio is below these thresholds, we prove that polynomial-time algorithms cannot solve these problems under the computational hardness conjectures of hypergraphic planted clique (HPC) detection and hypergraphic planted dense subgraph (HPDS) recovery. We also propose polynomial-time tensor algorithms that achieve reliable detection and recovery when the signal-to-noise ratio is above these thresholds. Both sparsity and tensor structures yield the computational barriers in high-order tensor clustering. The interplay between them results in significant differences between high-order tensor clustering and matrix clustering in literature in aspects of statistical and computational phase transition diagrams, algorithmic approaches, hardness conjecture, and proof techniques. To our best knowledge, we are the first to give a thorough characterization of the statistical and computational trade-off for such a double computational-barrier problem. In addition, we also provide evidence for the computational hardness conjectures of HPC detection and HPDS recovery.

show abstract

Section: Related Literaturementioning

confidence: 99%

Tensor Clustering with Planted Structures: Statistical Optimality and Computational Limits

Luo¹,

Zhang²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The first one is to design an optimization algorithm which would achieve a global convergence similarly to [45], but would work with the sparsity inducing penalty and possibly non-quadratic loss function. The second direction is to generalize results of [2] to general inductive matrix completion setting and to obtain a general minimax error bounds for the considered problem.…”

Section: Discussionmentioning

confidence: 99%

“…The combinatorial optimization algorithm that achieves fast recovery rates in the noisy case, is proposed in [39]. In the related setting of the inductive link prediction in graphs [2] the minimax optimal rates in sparse and low-rank regime are obtained and the tradeoffs between the statistical rates and the computational complexity are uncovered.…”

Section: Related Workmentioning

confidence: 99%

Sparse Group Inductive Matrix Completion

Nazarov¹,

Shirokikh²,

Maria³

et al. 2018

Preprint

View full text Add to dashboard Cite

We consider the problem of matrix completion with side information (inductive matrix completion). In real-world applications many side-channel features are typically non-informative making feature selection an important part of the problem. We incorporate feature selection into inductive matrix completion by proposing a matrix factorization framework with group-lasso regularization on side feature parameter matrices. We demonstrate, that the theoretical sample complexity for the proposed method is much lower compared to its competitors in sparse problems, and propose an efficient optimization algorithm for the resulting low-rank matrix completion problem with sparsifying regularizers. Experiments on synthetic and real-world datasets show that the proposed approach outperforms other methods.

show abstract

“…A number of average-case reductions in the literature have started with different average-case assumptions than the planted clique conjecture. Variants of planted dense subgraph have been used to show hardness in a model of financial derivatives under asymmetric information [ABBG11], link prediction [BB18], finding dense common subgraphs [CNW18] and online local learning of the size of a label set [ACLR15]. Hardness conjectures for random constraint satisfaction problems have been used to show hardness in improper learning complexity [DLSS14], learning DNFs [DSS16] and hardness of approximation [Fei02].…”

Section: Related Work On Statistical-computational Gapsmentioning

confidence: 99%

Optimal Average-Case Reductions to Sparse PCA: From Weak Assumptions to Strong Hardness

Brennan,

Bresler

2019

Preprint

View full text Add to dashboard Cite

In the past decade, sparse principal component analysis has emerged as an archetypal problem for illustrating statistical-computational tradeoffs. This trend has largely been driven by a line of research aiming to characterize the average-case complexity of sparse PCA through reductions from the planted clique (PC) conjecture -which conjectures that there is no polynomial-time algorithm to detect a planted clique of sizeAll of these reductions either fail to show tight computational lower bounds matching existing algorithms or show lower bounds for formulations of sparse PCA other than its canonical generative model, the spiked covariance model. Also, these lower bounds all quickly degrade with the exponent in the PC conjecture. Specifically, when only given the PC conjecture up to K = o(N α ) where α < 1/2, there is no sparsity level k at which these lower bounds remain tight. If α ≤ 1/3 these reductions fail to even show the existence of a statistical-computational tradeoff at any sparsity k. Our main results are as follows:• We give a reduction from PC that yields the first full characterization of the computational barrier in the spiked covariance model, providing tight lower bounds at all sparsities k. This partially resolves a question raised in [BBH18].• We show the surprising result that weaker forms of the PC conjecture up to clique size K = o(N α ) for any given α ∈ (0, 1/2] imply tight computational lower bounds for sparse PCA at sparsities k = o(n α/3 ). This shows that even a mild improvement in the signal strength needed by the best known polynomial-time sparse PCA algorithms would imply that the hardness threshold for PC is subpolynomial, rather than on the order N 1/2 as is widely conjectured.

show abstract

Optimal link prediction with matrix logistic regression

Cited by 6 publications

References 56 publications

Tensor Clustering with Planted Structures: Statistical Optimality and Computational Limits

Tensor Clustering with Planted Structures: Statistical Optimality and Computational Limits

Sparse Group Inductive Matrix Completion

Optimal Average-Case Reductions to Sparse PCA: From Weak Assumptions to Strong Hardness

Contact Info

Product

Resources

About