SISA: Set-Centric Instruction Set Architecture for Graph Mining on Processing-in-Memory Systems

Besta, Maciej; Raghavendra, K.; Kwasniewski, Grzegorz; Ausavarungnirun, Rachata; Beránek, Jakub; Kanellopoulos, Konstantinos; Janda, Kacper; Vonarburg-Shmaria, Zur; Gianinazzi, Lukas; Stefan, Ioana; Gómez-Luna, Juan; Golinowski, Jakub; Copik, Marcin; Kapp-Schwoerer, Lukas; Girolamo, Salvatore Di; Blach, Nils; Konieczny, Marek; Mutlu, Onur; Hoefler, Torsten

doi:10.1145/3466752.3480133

Cited by 29 publications

(12 citation statements)

References 135 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, one has to additionally compute graph clustering as a form of preprocessing. This can be parallelized with one of many established parallel clustering routines [29], [30], [36], [122].…”

Section: Appropriate Selection Of Target Verticesmentioning

confidence: 99%

“…Moreover, Field Programmable Gate Arrays (FPGAs) are well suited for problems that easily form pipelines. Finally, novel proposals include processing-in-memory (PIM) [29], [163] that brings computation closer to data.…”

Section: Single-machine Architecturesmentioning

confidence: 99%

“…• Incorporating new HW architectures While some initial works exist, there are not many designs on using GNNs with architectures such as FPGAs [35], [71], transactional processing [27], or processing in memory [4], [29], [88], [93], [98], [162], [163], [166], [182], [183].…”

Section: • Parallelization Of Gnn Models Beyond Simple C-gnnsmentioning

confidence: 99%

See 2 more Smart Citations

Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis

Besta¹,

Hoefler²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Graph neural networks (GNNs) are among the most powerful tools in deep learning. They routinely solve complex problems on unstructured networks, such as node classification, graph classification, or link prediction, with high accuracy. However, both inference and training of GNNs are complex, and they uniquely combine the features of irregular graph processing with dense and regular computations. This complexity makes it very challenging to execute GNNs efficiently on modern massively parallel architectures. To alleviate this, we first design a taxonomy of parallelism in GNNs, considering data and model parallelism, and different forms of pipelining. Then, we use this taxonomy to investigate the amount of parallelism in numerous GNN models, GNN-driven machine learning tasks, software frameworks, or hardware accelerators. We use the work-depth model, and we also assess communication volume and synchronization. We specifically focus on the sparsity/density of the associated tensors, in order to understand how to effectively apply techniques such as vectorization. We also formally analyze GNN pipelining, and we generalize the established Message-Passing class of GNN models to cover arbitrary pipeline depths, facilitating future optimizations. Finally, we investigate different forms of asynchronicity, navigating the path for future asynchronous parallel GNN pipelines. The outcomes of our analysis are synthesized in a set of insights that help to maximize GNN performance, and a comprehensive list of challenges and opportunities for further research into efficient GNN computations. Our work will help to advance the design of future GNNs.

show abstract

Section: Appropriate Selection Of Target Verticesmentioning

confidence: 99%

Section: Single-machine Architecturesmentioning

confidence: 99%

See 1 more Smart Citation

Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis

Besta¹,

Hoefler²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are two main approaches to PIM [4]: (1) processingnear-memory (PnM) [2,, where PIM logic is added to the same die as memory or to the logic layer of 3Dstacked memory [78][79][80]; and (2) processing-using-memory (PuM) [44,, which uses the operational principles of memory cells to perform computation (for example, by exploiting DRAM's analog operation to perform bulk bitwise AND, OR, and NOT logic operations [83,84,87,96,97,99]).…”

Section: Motivation and Problemmentioning

confidence: 99%

Methodologies, Workloads, and Tools for Processing-in-Memory: Enabling the Adoption of Data-Centric Architectures

F.¹,

Gómez-Luna²,

Ghose³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

“…These systems range from social networks [1], to biological networks [2], to the full Internet [3]. Data mining, information retrieval, recommendation systems, and fraud detection are just a few applications of graph analysis [4]- [8].…”

Section: Introductionmentioning

confidence: 99%

Asynchronous Distributed-Memory Triangle Counting and LCC with RMA Caching

Strausz¹,

Vella²,

Girolamo³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Triangle count and local clustering coefficient are two core metrics for graph analysis. They find broad application in analyses such as community detection and link recommendation. To cope with the computational and memory demands that stem from the size of today's graph datasets, distributedmemory algorithms have to be developed. Current state-of-theart solutions suffer from synchronization overheads or expensive pre-computations needed to distribute the graph, achieving limited scaling capabilities. We propose a fully asynchronous implementation for triangle counting and local clustering coefficient based on 1D partitioning, using remote memory accesses for transferring data and avoid synchronization. Additionally, we show how these algorithms present data reuse on remote memory accesses and how the overall communication time can be improved by caching these accesses. Finally, we extend CLaMPI, a software-layer caching system for MPI RMA, to include application-specific scores for cached entries and influence the eviction procedure to improve caching efficiency. Our results show improvements on shared memory, and we achieve 14x speedup from 4 to 64 nodes for the LiveJournal1 graph on distributed memory. Moreover, we demonstrate how caching remote accesses reduces total running time by up to 73% with respect to a non-cached version. Finally, we compare our implementation to TriC, the 2020 graph champion paper, and achieve up to 100x faster results for scale-free graphs.

show abstract

SISA: Set-Centric Instruction Set Architecture for Graph Mining on Processing-in-Memory Systems

Cited by 29 publications

References 135 publications

Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis

Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis

Methodologies, Workloads, and Tools for Processing-in-Memory: Enabling the Adoption of Data-Centric Architectures

Asynchronous Distributed-Memory Triangle Counting and LCC with RMA Caching

Contact Info

Product

Resources

About