Fast Sparse Matrix-Vector Multiplication by Exploiting Variable Block Structure

Abstract

We improve the performance of sparse matrix-vector multiply (SpMV) on modern cache-based superscalar machines when the matrix structure consists of multiple, irregularly aligned rectangular blocks. Matrices from finite element modeling applications often have this kind of structure. Our technique splits the matrix, A, into a sum, A 1 + A 2 + . . . + A s , where each term is stored in a new data structure, unaligned block compressed sparse row (UBCSR) format . The classical alternative approach of storing A in a block compressed sparse row (BCSR) format yields limited performance gains because it imposes a particular alignment of the matrix non-zero structure, leading to extra work from explicitly padded zeros. Combining splitting and UBCSR reduces this extra work while retaining the generally lower memory bandwidth requirements and register-level tiling opportunities of BCSR. Using application test matrices, we show empirically that speedups can be as high as 2.1× over not blocking at all, and as high as 1.8× over the standard BCSR implementation used in prior work. When performance does not improve, split UBCSR can still significantly reduce matrix storage.Through extensive experiments, we further show that the empirically optimal number of splittings s and the block size for each matrix term A i will in practice depend on the matrix and hardware platform. Our data lay a foundation for future development of fully automated methods for tuning these parameters.

Performance evaluation of the sparse matrix-vector multiplication on modern architectures

“…They experiment on a K6, a Power3, and an Itanium II processor for a suite of 20 sparse matrices and validate the accuracy of the proposed performance model. Vuduc et al [26] extend the notion of blocking in order to exploit variable block shapes by decomposing the original matrix to a proper sum of submatrices storing each submatrix in a variation of the BCSR format. Their approach is tested on the Ultra2i, Pentium III-M, Power4, and Itanium II processors for a suite of 10 FEM matrices that contain dense subblocks.…”

“…For example, blocking implemented with the use of the Block Compressed Storage Row (BCSR) format was proposed by Im and Yelick [11] as a transformation to tame irregular accesses on the input vector and exploit its inherent reuse, like in dense matrix optimizations. Onedimensional blocking is also proposed by Pinar and Heath [20] in order to reduce indirect memory references, while quite recently, Buttari et al [5] and Vuduc and Moon [26] accentuate the merit of blocking (the latter with variable-sized blocks) as a transformation to reduce indirect references and enable register level blocking and unrolling. However, it is not clarified if the benefits of blocking can be actually attributed to better cache utilization, memory access reduction, or ILP improvement.…”

“…The great importance and the singular performance behavior of SpMxV have attracted intense scientific attention [1,5,8,11,12,15,[17][18][19][20][23][24][25][26][27]. A general conclusion is that SpMxV can be efficiently optimized by exploiting information regarding the matrix structure and the processor's architectural characteristics.…”

“…Sparse matrix-vector computations and, in particular, sparse matrix-vector multiplication (SpMxV) have been recently categorized as one of the "seven dwarfs," i.e., seven numerical methods that are believed to be important for science and engineering for at least the next decade [2]. SpMxV is generally reported to perform poorly on modern microprocessors (e.g., 10% of peak performance [26]) due to a number of issues concerning the algorithm itself, the storage formats, and the sparsity patterns of the matrices.…”

A deeply-pipelined FPGA-based SpMV accelerator with a hardware-friendly storage scheme

“…However, as one of the "seven dwarfs" [1], SpMV is notorious for sustaining low fractions (less than 10% [2]) of the peak performance on the general purpose processors, mostly due to the inefficient use of the memory bandwidth. It results from the mismatches between the memory access patterns and the compression schemes of the sparse matrix.…”