yaSpMV

Abstract

SpMV is a key linear algebra algorithm and has been widely used in many important application domains. As a result, numerous attempts have been made to optimize SpMV on GPUs to leverage their massive computational throughput. Although the previous work has shown impressive progress, load imbalance and high memory bandwidth remain the critical performance bottlenecks for SpMV. In this paper, we present our novel solutions to these problems. First, we devise a new SpMV format, called blocked compressed common coordinate (BCCOO), which uses bit flags to store the row indices in a blocked common coordinate (COO) format so as to alleviate the bandwidth problem. We further improve this format by partitioning the matrix into vertical slices to enhance the cache hit rates when accessing the vector to be multiplied. Second, we revisit the segmented scan approach for SpMV to address the load imbalance problem. We propose a highly efficient matrix-based segmented sum/scan for SpMV and further improve it by eliminating global synchronization. Then, we introduce an auto-tuning framework to choose optimization parameters based on the characteristics of input sparse matrices and target hardware platforms. Our experimental results on GTX680 GPUs and GTX480 GPUs show that our proposed framework achieves significant performance improvement over the vendor tuned CUSPARSE V5.0 (up to 229% and 65% on average on GTX680 GPUs, up to 150% and 42% on average on GTX480 GPUs) and some most recently proposed schemes (e.g., up to 195% and 70% on average over clSpMV on GTX680 GPUs, up to 162% and 40% on average over clSpMV on GTX480 GPUs).

Speculative segmented sum for sparse matrix-vector multiplication on heterogeneous processors

“…Because of less off-chip memory access and better on-chip memory localization, block-based formats or libraries, such as OSKI [38,42,43], pOSKI [37], CSB [44,45], BELLPACK [46], BCCOO/BCCOO+ [5], BRC [6] and RSB [47], attracted the most attention. However, block-based formats heavily rely on sparsity structure, meaning that the input matrix is required to have a block structure to meet potential block layout.…”

“…Table 3 lists main information of the evaluated sparse matrices. The first 14 matrices of the benchmark suite have been widely used in previous work [1,2,5,6,12,13]. The last 6 matrices are chosen as representatives of irregular matrices extracted from graph applications, such as circuit simulation and optimization problems.…”

“…Thus a lot of research has focused on accelerating SpMV on co-processors. One straightforward way on utilizing co-processors is to develop all-new sparse matrix formats (e.g., HYB [1], Cocktail [2], JAD [3], ESB [4], BCCOO [5] and BRC [6]) for specific hardware architectures. The experimental results showed that these formats can provide performance improvement for various SpMV benchmarks.…”

Optimization techniques for sparse matrix–vector multiplication on GPUs

“…Focusing on pure performance, the state-of-the-art in SpMV optimization is arguably BCCOO [28]. This advanced sparse matrix format is an evolution of COO based on blocking and row indices compression, where load balancing is achieved by the means of a highly-efficient segmented reduction (which, however, relies on a non-portable synchronization-free mechanism that stalls on modern AMD GPUs).…”

“…The first column of the Table 16 reports the time necessary to load the original Matrix Market file [33] from disk. The following columns are instead the preprocessing times to compose and tune CSR, AdELL+ and BCCOO [28]. Note that the measurements for this latter format are available only in single-precision due to the lack of double-precision implementation.…”