ReGraphX: NoC-enabled 3D Heterogeneous ReRAM Architecture for Training Graph Neural Networks

Abstract

Graph Neural Network (GNN) is a variant of Deep Neural Networks (DNNs) operating on graphs. However, GNNs are more complex compared to traditional DNNs as they simultaneously exhibit features of both DNN and graph applications. As a result, architectures specifically optimized for either DNNs or graph applications are not suited for GNN training. In this work, we propose a 3D heterogeneous manycore architecture for on-chip GNN training to address this problem. The proposed architecture, ReGraphX, involves heterogeneous ReRAM crossbars to fulfill the disparate requirements of both DNN and graph computations simultaneously. The ReRAMbased architecture is complemented with a multicast-enabled 3D NoC to improve the overall achievable performance. We demonstrate that ReGraphX outperforms conventional GPUs by up to 3.5X (on an average 3X) in terms of execution time, while reducing energy consumption by as much as 11X.

Enabling Neuromorphic Computing for Artificial Intelligence with Hardware-Software Co-Design

“…Subsequently, Spara presented a novel vertex mapping strategy to address this challenge [52]. There are also ReRAM-based architectures for graph processing focus on sparsity [53,54], three-dimensional architecture [55,56], regularization, redundant computation [57], etc. Transformers, one of the most advanced models in current natural language processing (NLP), present several challenges to the ReRAM-based neuromorphic computing [93].…”

“…These accelerators demonstrated improved speed and energy efficiency compared to traditional computing platforms such as CPUs and GPUs. With the continuous advancement of ReRAM technology, the ReRAM-based neuromorphic engines are being applied in broader domains [48][49][50][51][52][53][54][55][56][57][58][59][60][61].…”

COIN: Communication-Aware In-Memory Acceleration for Graph Convolutional Networks

“…However, these technique does not address the issue of the on-chip communication performance of GCN accelerators. Recently, a RRAM-based 3D NoC-enabled accelerator for GNN training ReGraphX is proposed [16]. The authors show that the proposed architecture is more energy-efficient than conventional GPUs.…”

“…This section compares the performance of our proposed COIN architecture with a state-of-the-art GCN accelerator, ReGraphX [16] and AWB-GCN [15]. Comparison with ReGraphX [16]: The architecture proposed in ReGraphX is composed of multiple processing elements (PEs; similar to computing elements in COIN). Some of the PEs (V-PEs) store the weights and are responsible for the feature extraction operation at GCN nodes (or vertices).…”

“…The prevalence and computational complexity of GCNs call for high-performance and energy-efficient hardware accelerators. In contrast to software implementations, hardware accelerators perform GCN computations with significantly lower latency and higher energy efficiency [14][15][16]. Due to this potential, a couple of recent studies proposed GCN accelerators [14,17].…”