Computing Graph Neural Networks: A Survey from Algorithms to Accelerators

Abstract

Graph Neural Networks (GNNs) have exploded onto the machine learning scene in recent years owing to their capability to model and learn from graph-structured data. Such an ability has strong implications in a wide variety of fields whose data are inherently relational, for which conventional neural networks do not perform well. Indeed, as recent reviews can attest, research in the area of GNNs has grown rapidly and has lead to the development of a variety of GNN algorithm variants as well as to the exploration of ground-breaking applications in chemistry, neurology, electronics, or communication networks, among others. At the current stage research, however, the efficient processing of GNNs is still an open challenge for several reasons. Besides of their novelty, GNNs are hard to compute due to their dependence on the input graph, their combination of dense and very sparse operations, or the need to scale to huge graphs in some applications. In this context, this article aims to make two main contributions. On the one hand, a review of the field of GNNs is presented from the perspective of computing. This includes a brief tutorial on the GNN fundamentals, an overview of the evolution of the field in the last decade, and a summary of operations carried out in the multiple phases of different GNN algorithm variants. On the other hand, an in-depth analysis of current software and hardware acceleration schemes is provided, from which a hardware-software, graph-aware, and communication-centric vision for GNN accelerators is distilled.

FlowGNN: A Dataflow Architecture for Universal Graph Neural Network Inference via Multi-Queue Streaming

“…GNN acceleration is attracting intensive attention in the research community. Recent works are summarized by a survey [1], including accelerations using CPU/GPUs, ASICs, FPGAs, and heterogeneous platforms. Auten et al [3] propose the first GNN accelerator composed of four modules: graph traversal, matrix operation, data scheduling, and graph aggregations.…”

GROW: A Row-Stationary Sparse-Dense GEMM Accelerator for Memory-Efficient Graph Convolutional Neural Networks

“…There is a large body of prior work that seek acceleration of GCNs [1], [3], [9], [18], [19], [23], [24], [33], [34], [43], [45], [47], [48], [50]- [53]. Section II-C discussed the most relevant GCN accelerators that we directly compare against GROW.…”

Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

“…A few reviews [Wu et al, 2020;Battaglia et al, 2018] pay close attention to GNN models and generic applications, while others [Wang et al, 2021b; place emphasis on specific usages of GNNs. Moreover, hardware-related architectures [Abadal et al, 2021;Han et al, 2021] and software-related algorithms [Lamb et al, 2020] of GNNs are also emphatically surveyed by researchers. Thereby, the above reviews further promote the widespread use of GNNs.…”