Clustered low rank approximation of graphs in information science applications

Savas, Berkant; Dhillon, Inderjit S.

doi:10.1137/1.9781611972818.15

Cited by 43 publications

(47 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This algorithm may be viewed as post-processing the relative-error random sampling algorithm of the previous subsection to remove redundant columns; and it has been applied successfully to a range of data analysis problems. See, e.g., [26,27,28] and [155,156,157], as well as [158] for a discussion of numerical issues associated with this algorithm.…”

Section: A Two-stage Hybrid Algorithm For This Problemmentioning

confidence: 99%

Randomized Algorithms for Matrices and Data

Mahoney¹

2012

Advances in Machine Learning and Data Mining for Astronomy

327

373

View full text Add to dashboard Cite

Randomized algorithms for very large matrix problems have received a great deal of attention in recent years. Much of this work was motivated by problems in large-scale data analysis, largely since matrices are popular structures with which to model data drawn from a wide range of application domains, and this work was performed by individuals from many different research communities. While the most obvious benefit of randomization is that it can lead to faster algorithms, either in worst-case asymptotic theory and/or numerical implementation, there are numerous other benefits that are at least as important. For example, the use of randomization can lead to simpler algorithms that are easier to analyze or reason about when applied in counterintuitive settings; it can lead to algorithms with more interpretable output, which is of interest in applications where analyst time rather than just computational time is of interest; it can lead implicitly to regularization and more robust output; and randomized algorithms can often be organized to exploit modern computational architectures better than classical numerical methods.This monograph will provide a detailed overview of recent work on the theory of randomized matrix algorithms as well as the application of those ideas to the solution of practical problems in large-scale data analysis. Throughout this review, an emphasis will be placed on a few simple core ideas that underlie not only recent theoretical advances but also the usefulness of these tools in large-scale data applications. Crucial in this context is the connection with the concept of statistical leverage. This concept has long been used in statistical regression diagnostics to identify outliers; and it has recently proved crucial in the development of improved worst-case matrix algorithms that are also amenable to high-quality numerical implementation and that are useful to domain scientists. This connection arises naturally when one explicitly decouples the effect of randomization in these matrix algorithms from the underlying linear algebraic structure. This decoupling also permits much finer control in the application of randomization, as well as the easier exploitation of domain knowledge.Most of the review will focus on random sampling algorithms and random projection algorithms for versions of the linear least-squares problem and the low-rank matrix approximation problem. These two problems are fundamental in theory and ubiquitous in practice. Randomized methods solve these problems by constructing and operating on a randomized sketch of the input matrix Afor random sampling methods, the sketch consists of a small number of carefully-sampled and rescaled columns/rows of A, while for random projection methods, the sketch consists of a small number of linear combinations of the columns/rows of A. Depending on the specifics of the situation, when compared with the best previously-existing deterministic algorithms, the resulting randomized algorithms have worst-case running time that is asympt...

show abstract

Section: A Two-stage Hybrid Algorithm For This Problemmentioning

confidence: 99%

Randomized Algorithms for Matrices and Data

Mahoney¹

2012

Advances in Machine Learning and Data Mining for Astronomy

327

373

View full text Add to dashboard Cite

show abstract

“…In this section, we describe the clustered low-rank approximation method proposed in [25], and introduce the problem of link prediction in social network analysis. Throughout the paper, we use capital letters to represent matrices, lower-case bold letters to represent vectors, and lower-case italics to represent scalars.…”

Section: Preliminariesmentioning

confidence: 99%

“…There are a number of benefits of clustered low rank approximation compared to spectral regular low-rank approximation: (1) the clustered low-rank approximation preserves important structural information of a network by extracting a certain amount of information from all of the clusters; (2) it has been shown that the clustered low-rank approximation achieves a lower relative error than the truncated SVD with the same amount of memory [25]; (3) it also has been shown that even a sequential implementation of clustered low rank approximation [25] is faster than state-of-the-art algorithms for low-rank matrix approximation [20]; (4) improved accuracy of clustered low-rank approximation contributes to improved performance of end tasks, e.g., prediction of new links in social networks [26] and group recommendation to community members [29].…”

Section: Clustered Low-rank Approximationmentioning

confidence: 99%

“…We need to extract some information from every cluster regardless of its size to preserve important structural information of the original network in a low-dimensional representation. This is the motivation of a recently proposed method called clustered low-rank approximation [25], which reflects the clustering structure of the original network in the low-rank representation of the network. It extracts clusters, computes a low-rank approximation of each cluster, and then combines together the cluster approximations to approximate the entire network.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Parallel Clustered Low-Rank Approximation of Graphs and Its Application to Link Prediction

Sui

Lee

Whang

et al. 2013

Languages and Compilers for Parallel Computing

Self Cite

View full text Add to dashboard Cite

Abstract. Social network analysis has become a major research area that has impact in diverse applications ranging from search engines to product recommendation systems. A major problem in implementing social network analysis algorithms is the sheer size of many social networks, for example, the Facebook graph has more than 900 million vertices and even small networks may have tens of millions of vertices. One solution to dealing with these large graphs is dimensionality reduction using spectral or SVD analysis of the adjacency matrix of the network, but these global techniques do not necessarily take into account local structures or clusters of the network that are critical in network analysis. A more promising approach is clustered low-rank approximation: instead of computing a global low-rank approximation, the adjacency matrix is first clustered, and then a low-rank approximation of each cluster (i.e., diagonal block) is computed. The resulting algorithm is challenging to parallelize not only because of the large size of the data sets in social network analysis, but also because it requires computing with very diverse data structures ranging from extremely sparse matrices to dense matrices. In this paper, we describe the first parallel implementation of a clustered low-rank approximation algorithm for large social network graphs, and use it to perform link prediction in parallel. Experimental results show that this implementation scales well on large distributed-memory machines; for example, on a Twitter graph with roughly 11 million vertices and 63 million edges, our implementation scales by a factor of 86 on 128 processes and takes less than 2300 seconds, while on a much larger Twitter graph with 41 million vertices and 1.2 billion edges, our implementation scales by a factor of 203 on 256 processes with a running time about 4800 seconds.

show abstract

“…Note that with c = 1, CSGE becomes the regular SGE. In related preliminary evaluation, we explored the benet of combining clustering with different local (within-cluster) low rank approximation schemes from the viewpoint of numerical accuracy on a few static graphs [38]. However, higher numerical accuracy in matrix approximations does not necessarily translate to benets in end applications, such as link prediction.…”

Section: Basic Algorithmmentioning

confidence: 99%

Clustered embedding of massive social networks

Song

Savas

Cho

et al. 2012

SIGMETRICS Perform. Eval. Rev.

Self Cite

View full text Add to dashboard Cite

-The explosive growth of social networks has created numerous exciting research opportunities. A central concept in the analysis of social networks is a proximity measure, which captures the closeness or similarity between nodes in the network. Despite much research on proximity measures, there is a lack of techniques to efciently and accurately compute proximity measures for largescale social networks. In this paper, we embed the original massive social graph into a much smaller graph, using a novel dimensionality reduction technique termed Clustered Spectral Graph Embedding. We show that the embedded graph captures the essential clustering and spectral structure of the original graph and allow a wide range of analysis to be performed on massive social graphs. Applying the clustered embedding to proximity measurement of social networks, we develop accurate, scalable, and exible solutions to three important social network analysis tasks: proximity estimation, missing link inference, and link prediction. We demonstrate the effectiveness of our solutions to the tasks in the context of large real-world social network datasets: Flickr, LiveJournal, and MySpace with up to 2 million nodes and 90 million links.

show abstract

Clustered low rank approximation of graphs in information science applications

Cited by 43 publications

References 31 publications

Randomized Algorithms for Matrices and Data

Randomized Algorithms for Matrices and Data

Parallel Clustered Low-Rank Approximation of Graphs and Its Application to Link Prediction

Clustered embedding of massive social networks

Contact Info

Product

Resources

About