Register-Aware Optimizations for Parallel Sparse Matrix–Matrix Multiplication

Abstract

General sparse matrix-matrix multiplication (SpGEMM) is a fundamental building block of a number of high-level algorithms and real-world applications. In recent years, several efficient SpGEMM algorithms have been proposed for many-core processors such as GPUs. However, their implementations of sparse accumulators, the core component of SpGEMM, mostly use low speed on-chip shared memory and global memory, and high speed registers are seriously underutilised. In this paper, we propose three novel register-aware SpGEMM algorithms for three representative sparse accumulators, i.e., sort, merge and hash, respectively. We fully utilise the GPU registers to fetch data, finish computations and store results out. In the experiments, our algorithms deliver excellent performance on a benchmark suite including 205 sparse matrices from the SuiteSparse Matrix Collection. Specifically, on an Nvidia Pascal P100 GPU, our three register-aware sparse accumulators achieve on average 2.0x (up to 5.4x), 2.6x (up to 10.5x) and 1.7x (up to 5.2x) speedups over their original implementations in libraries bhSPARSE, RMerge and NSPARSE, respectively.

FCNNLib: A Flexible Convolution Algorithm Library for Deep Learning on FPGAs

“…Most of these libraries include heterogeneous convolution algorithms and provide primitives that help in algorithm selection. In addition, several of previously proposed effective optimizations for machine learning kernels on CPUs and GPUs [57], [58], [59], [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70] can be potentially integrated into libraries.…”

Distributed-memory parallel algorithms for sparse times tall-skinny-dense matrix multiplication

“…However, SpMM attains a significantly higher fraction of peak on local compute nodes, hence completes its local computation steps faster. For example, the reported performance rate for SpMM on the NVIDIA P100 GPU range in 100−500 GFlops [17], yet SpGEMM can only achieve 1−10 GFlops on the same GPU [25]. Consequently, SpGEMM more effectively hides its communication costs with local computation, compared to SpMM.…”

Bandwidth-Optimized Parallel Algorithms for Sparse Matrix-Matrix Multiplication using Propagation Blocking

“…For output formations, we have many options. Prior work used the expand-sort-compress strategy [15], [24], [18] or used accumulators based on heap [22], hash table [12], and a dense vector called SPA [20], [25]. Table I summarizes prior work based on their data access patterns.…”

“…After the entire Ĉ is constructed, flop tuples in Ĉ are sorted and merged to generate the final output. Since sorting can be efficiently performed on GPUs, ESC SpGEMM can performed better than other algorithms on GPUs [15], [18]. The column ESC algorithm has access patterns for A, B, and C similar to the column SpGEMM algorithm.…”

“…However, most practical SpGEMM operations have small compression factors. For example, when squaring matrices from the SuiteSparse Matrix Collection, more than 80% of SpGEMMs have a compression factor less than 3 and about 99% have a compression factor less than 6 [18]. Hence, for most practical scenarios, PB-SpGEMM performs predictably better than existing heap and hash algorithms.…”