FusedGCN: A Systolic Three-Matrix Multiplication Architecture for Graph Convolutional Networks

Abstract

Systolic Array (SA) architectures are well suited for accelerating matrix multiplications through the use of a pipelined array of Processing Elements (PEs) communicating with local connections and pre-orchestrated data movements. Even though most of the dynamic power consumption in SAs is due to multiplications and additions, pipelined data movement within the SA constitutes an additional important contributor. The goal of this work is to reduce the dynamic power consumption associated with the feeding of data to the SA, by synergistically applying bus-invert coding and zero-value clock gating. By exploiting salient attributes of state-of-the-art CNNs, such as the value distribution of the weights, the proposed SA applies appropriate encoding only to the data that exhibits high switching activity. Similarly, when one of the inputs is zero, unnecessary operations are entirely skipped. This selectively targeted, application-aware encoding approach is demonstrated to reduce the dynamic power consumption of data streaming in CNN applications using Bfloat16 arithmetic by 1%-19%. This translates to an overall dynamic power reduction of 6.2%-9.4%.

Exploiting Neural-Network Statistics for Low-Power DNN Inference

“…Recent research already investigated low-power coding to decrease the power consumption in DNNs accelerators [14]. In particular, it uses bus-invert for coding the mantissa of the bfloat16 numbers used in a systolic array.…”

“…Second, Ref. [14] focuses only on the data-path of the core accelerator, disregarding potential savings in the memories and interconnects-which however typically exhibit the highest power needs. Moreover, Ref.…”

“…Moreover, Ref. [14] focuses on floating point numbers, while AI inference at the low-power edge is dominantly uses integer quantization [2]. Thus, this work presents the first, holistic and application-specific low-power coding approach for integer-quantized DNN inference.…”

Templatized Fused Vector Floating-Point Dot Product for High-Level Synthesis

“…Matrix multiplications are at the heart of deep learning algorithms and their computation in hardware maps naturally onto Systolic Arrays (SA) [5]. Tensor processing units [6] and other related architectures [7]- [10] are characteristic examples of newly designed SAs.…”