Characterizing and Understanding GCNs on GPU

Abstract

Graph convolutional neural networks (GCNs) have achieved state-of-the-art performance on graph-structured data analysis. Like traditional neural networks, training and inference of GCNs are accelerated with GPUs. Therefore, characterizing and understanding the execution pattern of GCNs on GPU is important for both software and hardware optimization. Unfortunately, to the best of our knowledge, there is no detailed characterization effort of GCN workloads on GPU. In this paper, we characterize GCN workloads at inference stage and explore GCN models on NVIDIA V100 GPU. Given the characterization and exploration, we propose several useful guidelines for both software optimization and hardware optimization for the efficient execution of GCNs on GPU.

Computing Graph Neural Networks: A Survey from Algorithms to Accelerators

“…GNN computing presents a set of unique challenges [163,182] that have rendered existing libraries and hardware platforms inefficient, including:…”

“…aggregation can be done via sparse GEMM of the adjacency matrix [163], but they are not generalizable to all graphs/GNNs and typically not enough to combat the extreme sparsity of adjacency matrices. Therefore, the challenge is to develop architectures that accelerate such distinct phases and their intertwining at runtime.…”

“…A direct consequence of the aforementioned aspects is that the bottleneck or the critical operation/kernel may vary across GNNs or applications, as shown in References [10,163,182]. In light of these challenges, GNNs call for new solutions both in software and hardware.…”

DeepBurning-GL

“…These two stages operating on a large and sparse graph can incur dynamic computational data flow and numerous irregular memory access. The coexistence of regular neural network operations and irregular graph level operations in GNN does not favor conventional CPU and GPU solutions showing low efficiency when facing the divergent computation patterns of the four stages [25,29].…”

fuseGNN

“…In recent years, graph convolutional neural networks (GNN) that operate on graph-structured data have achieved convincing performance on tasks like node and graph classification [10,23,24].…”