Executing Dynamic Data-Graph Computations Deterministically Using Chromatic Scheduling

Kaler, Tim; Hasenplaugh, William; Schardl, Tao B.; Leiserson, Charles E.

doi:10.1201/9781003033707-18

Cited by 5 publications

(5 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, in this work, we focus on minibatch training. For minibatch training, existing systems include Dist-DGL (Zheng et al, 2020), Quiver, GNNLab (Yang et al, 2022c), WholeGraph (Yang et al, 2022b), DSP (Cai et al, 2023), PGLBox (Jiao et al, 2023), SALIENT++ (Kaler et al, 2023), NextDoor (Jangda et al, 2021), P 3 (Gandhi & Iyer, 2021). Here, the main performance bottleneck is the cost of sampling minibatches.…”

Section: Distributed Gnn Systemsmentioning

confidence: 99%

Reducing Communication in Graph Neural Network Training

Tripathy

Yelick

Buluç

2020

SC20: International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) are powerful and flexible neural networks that use the naturally sparse connectivity information of the data. GNNs represent this connectivity as sparse matrices, which have lower arithmetic intensity and thus higher communication costs compared to dense matrices, making GNNs harder to scale to high concurrencies than convolutional or fully-connected neural networks.We present a family of parallel algorithms for training GNNs. These algorithms are based on their counterparts in dense and sparse linear algebra, but they had not been previously applied to GNN training. We show that they can asymptotically reduce communication compared to existing parallel GNN training methods. We implement a promising and practical version that is based on 2D sparse-dense matrix multiplication using torch.distributed. Our implementation parallelizes over GPU-equipped clusters. We train GNNs on up to a hundred GPUs on datasets that include a protein network with over a billion edges.

show abstract

Section: Distributed Gnn Systemsmentioning

confidence: 99%

Reducing Communication in Graph Neural Network Training

Tripathy

Yelick

Buluç

2020

SC20: International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

show abstract

“…It allows us to offer a feasible solution and establishes lower bound results for EVG. In practice, they can be supported by invoking established solutions, e.g., parallel GNN inference [19,31] and subgraph pattern matching [23,46], respectively. Pattern generators.…”

Section: Generating Explanation Viewsmentioning

confidence: 99%

View-based Explanations for Graph Neural Networks

Chen,

Qiu,

et al. 2024

Proc. ACM Manag. Data

View full text Add to dashboard Cite

Generating explanations for graph neural networks (GNNs) has been studied to understand their behaviors in analytical tasks such as graph classification. Existing approaches aim to understand the overall results of GNNs rather than providing explanations for specific class labels of interest, and may return explanation structures that are hard to access, nor directly queryable. We propose GVEX, a novel paradigm that generates Graph Views for GNN EXplanation. (1) We design a two-tier explanation structure called explanation views. An explanation view consists of a set of graph patterns and a set of induced explanation subgraphs. Given a database G of multiple graphs and a specific class label l assigned by a GNN-based classifier M, it concisely describes the fraction of G that best explains why l is assigned by M. (2) We propose quality measures and formulate an optimization problem to compute optimal explanation views for GNN explanation. We show that the problem is Σ2P-hard. (3) We present two algorithms. The first one follows an explain-and-summarize strategy that first generates high-quality explanation subgraphs which best explain GNNs in terms of feature influence maximization, and then performs a summarization step to generate patterns. We show that this strategy provides an approximation ratio of 1/2. Our second algorithm performs a single-pass to an input node stream in batches to incrementally maintain explanation views, having an anytime quality guarantee of 1/4-approximation. Using real-world benchmark data, we experimentally demonstrate the effectiveness, efficiency, and scalability of GVEX. Through case studies, we showcase the practical applications of GVEX.

show abstract

“…Efficient subgraph extraction is the main direction of recent system works to scale SGRL models. These techniques include PPR-based [4,52] and random walk-based [51] subgraph samplers, node neighborhood sampling through CUDA kernel (DGL, [11]), tensor operations (PyG, [36]), and performanceengineered sampler (SALIENT, [21]), as well as parallel sampling for temporal graphs [58]. Some frameworks also customize data structures to better support subgraph operations and gain higher throughput, such as associative arrays in SUREL [51], temporal-CSR in TGL [58] and GPU-orientated dictionary in NAT [32].…”

Section: Related Workmentioning

confidence: 99%

SUREL+: Moving from Walks to Sets for Scalable Subgraph-based Graph Representation Learning

Yin¹,

Zhang²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Subgraph-based graph representation learning (SGRL) has recently emerged as a powerful tool in many prediction tasks on graphs due to its advantages in model expressiveness and generalization ability. Most previous SGRL models face computational issues associated with the high cost of extracting subgraphs for each training or testing query. Recently, SUREL has been proposed as a new framework to accelerate SGRL, which samples random walks offline and joins these walks as subgraphs online for prediction. Due to the reusability of sampled walks across different queries, SUREL achieves state-of-the-art performance in both scalability and prediction accuracy. However, SUREL still suffers from high computational overhead caused by node redundancy in sampled walks. In this work, we propose a novel framework SUREL+ that upgrades SUREL by using node sets instead of walks to represent subgraphs. This set-based representation avoids node duplication by definition, but the sizes of node sets can be irregular. To address this issue, we design a dedicated sparse data structure to efficiently store and fast index node sets, and provide a specialized operator to join them in parallel batches. SUREL+ is modularized to support multiple types of set samplers, structural features, and neural encoders to complement the loss of structural information due to the reduction from walks to sets. Extensive experiments have been performed to validate SUREL+ in the prediction tasks of links, relation types, and higher-order patterns. SUREL+ achieves 3-11× speedups of SUREL while maintaining comparable or even better prediction performance; compared to other SGRL baselines, SUREL+ achieves ∼20× speedups and significantly improves the prediction accuracy.

show abstract

Executing Dynamic Data-Graph Computations Deterministically Using Chromatic Scheduling

Cited by 5 publications

References 72 publications

Reducing Communication in Graph Neural Network Training

Reducing Communication in Graph Neural Network Training

View-based Explanations for Graph Neural Networks

SUREL+: Moving from Walks to Sets for Scalable Subgraph-based Graph Representation Learning

Contact Info

Product

Resources

About