Graph Sparsification, Spectral Sketches, and Faster Resistance Computation via Short Cycle Decompositions

Chu, Timothy; Gao, Yu; Peng, Richard; Sachdeva, Sushant; Sawlani, Saurabh; Wang, Junxing

doi:10.1137/19m1247632

Cited by 11 publications

(9 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work, we use spectral sparsification to choose a randomized subgraph. Apart form providing the strongest guarantees in preserving graph structure [25], they align well with GNNs due to their connection to spectral graph convolutions. Our work is similar to [26], which uses spectral graph sparsification to accelerate simpler Laplacian smoothing based statistical inference tasks such as regression.…”

Section: Related Workmentioning

confidence: 93%

Fast Graph Attention Networks Using Effective Resistance Based Graph Sparsification

Srinivasa,

Xiao,

Glass

et al. 2020

Preprint

View full text Add to dashboard Cite

The attention mechanism has demonstrated superior performance for inference over nodes in graph neural networks (GNNs), however, they result in a high computational burden during both training and inference. We propose FastGAT, a method to make attention based GNNs lightweight by using spectral sparsification to generate an optimal pruning of the input graph. This results in a per-epoch time that is almost linear in the number of graph nodes as opposed to quadratic. Further, we provide a re-formulation of a specific attention based GNN, Graph Attention Network (GAT) that interprets it as a graph convolution method using the random walk normalized graph Laplacian. Using this framework, we theoretically prove that spectral sparsification preserves the features computed by the GAT model, thereby justifying our FastGAT algorithm. We experimentally evaluate FastGAT on several large real world graph datasets for node classification tasks, FastGAT can dramatically reduce (up to 10x) the computational time and memory requirements, allowing the usage of attention based GNNs on large graphs.

show abstract

Section: Related Workmentioning

confidence: 93%

Fast Graph Attention Networks Using Effective Resistance Based Graph Sparsification

Srinivasa,

Xiao,

Glass

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Graph Sparsification aims to approximate a graph on the same set of vertices and edges (Benczur & Karger, 2002). Much research has been done in this field and several algorithms were proposed, including spectral sparsifiers (Spielman & Teng, 2008;Calandriello et al, 2018;Chu et al, 2020), sampling via Metropolis algorithms (Hübler et al, 2008), and others (Sadhanala et al, 2016). In the context of training graph neural networks, the motivation is twofold: 1. accelerate training by performing fewer message-passing operations, and 2. regularization.…”

Section: Related Workmentioning

confidence: 99%

LSP : Acceleration and Regularization of Graph Neural Networks via Locality Sensitive Pruning of Graphs

Kosman¹,

Oren²,

Castro³

2021

Preprint

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have emerged as highly successful tools for graph-related tasks. However, real-world problems involve very large graphs, and the compute resources needed to fit GNNs to those problems grow rapidly. Moreover, the noisy nature and size of real-world graphs cause GNNs to over-fit if not regularized properly. Surprisingly, recent works show that large graphs often involve many redundant components that can be removed without compromising the performance too much. This includes node or edge removals during inference through GNNs layers or as a pre-processing step that sparsifies the input graph. This intriguing phenomenon enables the development of state-of-the-art GNNs that are both efficient and accurate. In this paper, we take a further step towards demystifying this phenomenon and propose a systematic method called Locality-Sensitive Pruning (LSP) for graph pruning based on Locality-Sensitive Hashing. We aim to sparsify a graph so that similar local environments of the original graph result in similar environments in the resulting sparsified graph, which is an essential feature for graph-related tasks. To justify the application of pruning based on local graph properties, we exemplify the advantage of applying pruning based on locality properties over other pruning strategies in various scenarios. Extensive experiments on synthetic and real-world datasets demonstrate the superiority of LSP, which removes a significant amount of edges from large graphs without compromising the performance, accompanied by a considerable acceleration.

show abstract

“…Graph Sparsification. Graph sparsification was introduced by Benczúr and Karger [8] ("for-all" cut sparsfiers), and has led to research in a number of directions: Fung et al [17] and Kapralov and Panigrahy [27] gave new algorithms for preserving cuts in a sparsifier; Spielman and Teng [46] generalized to spectral sparsfiers that preserved all quadratic forms, which led to further research both in improving the bounds on the size of the sparsifier [45,7] and also in the running time of spectral sparsification algorithms (e.g., [35,5,36,11,34,31,33,32]); faster algorithms for fundamental graph problems such as maximum flow utilized sparsification results (e.g., [8,43]); Ahn and Guha [1] introduced sparsification in the streaming model, which has led to a large body of work for both cut sparifiers (e.g., [2,3,18]) and spectral sparsifiers (e.g., [26,25,24,4]) in graph streams; both cut [30,40] and spectral [44] sparsification have been studied in hypergraphs; etc. For lower bounds, Andoni et al [6] showed that any data structure that (1± )-approximately stores the sizes of all cuts in an undirected graph must use Ω(n/ 2 ) bits.…”

Section: Related Workmentioning

confidence: 99%

Sparsification of Directed Graphs via Cut Balance

Cen,

Cheng,

Panigrahi

et al. 2020

Preprint

View full text Add to dashboard Cite

Sparsification, where the cut values of an input graph are approximately preserved by a sparse graph (called a cut sparsifier) or a succinct data structure (called a cut sketch), has been an influential tool in graph algorithms. But, this tool is restricted to undirected graphs, because some directed graphs are known to not admit sparsification. Such examples, however, are structurally very dissimilar to undirected graphs in that they exhibit highly unbalanced cuts. This motivates us to ask: can we sparsify a balanced digraph?To make this question concrete, we define balance β of a digraph as the maximum ratio of the cut value in the two directions (Ene et al., STOC 2016). We show the following results: * Part of this work was done when the author was a postdoctoral researcher at Duke University and when he was visiting the Institute of Advanced Study.

show abstract

Graph Sparsification, Spectral Sketches, and Faster Resistance Computation via Short Cycle Decompositions

Cited by 11 publications

References 48 publications

Fast Graph Attention Networks Using Effective Resistance Based Graph Sparsification

Fast Graph Attention Networks Using Effective Resistance Based Graph Sparsification

LSP : Acceleration and Regularization of Graph Neural Networks via Locality Sensitive Pruning of Graphs

Sparsification of Directed Graphs via Cut Balance

Contact Info

Product

Resources

About