Predicting an Optimal Sparse Matrix Format for SpMV Computation on GPU

Abstract

Many-threaded architecture based Graphics Processing Units (GPUs) are good for general purpose computations for achieving high performance. The processor has latency hiding mechanism through which it hides the memory access time in such a way that when one warp (group of 32 threads) is computing, the other warps perform memory bound access. But for memory access bound irregular applications such as Sparse Matrix Vector Multiplication (SpMV), memory access times are high and hence improving the performance of such applications on GPU is a challenging research issue. Further, optimizing SpMV time on GPU is an important task for iterative applications like jacobi and conjugate gradient. However, there is a need to consider the overheads caused while computing SpMV on GPU. Transforming the input matrix to a desired format and communicating the data from CPU to GPU are non-trivial overheads associated with SpMV computation on GPU. If the chosen format is not suitable for the given input sparse matrix then desired performance improvements cannot be achieved.Motivated by this observation, this paper proposes a method to chose an optimal sparse matrix format, focusing on the applications where CPU to GPU communication time and preprocessing time are nontrivial. The experimental results show that the predicted format by the model matches with that of the actual high performing format when total SpMV time in terms of pre-processing time, CPU to GPU communication time and SpMV computation time on GPU, is taken into account. The model predicts an optimal format for any given input sparse matrix with a very small overhead of prediction within an application. Compared to the format to achieve high performance only on GPU, our approach is more comprehensive and valuable. This paper also proposes to use a communication and pre-processing overhead optimizing sparse matrix format to be used when these overheads are non trivial.

Autotuning Runtime Specialization for Sparse Matrix-Vector Multiplication

“…We leave it for future work to see whether this approach improves the prediction accuracy for our experiments. Most of the other autotuning work have smaller matrix sets than ours, for example, ∼14-150 [Grewe and Lokhmotov 2011;Muralidharan et al 2014;Neelima et al 2014;Guo et al 2014]. There also are studies with bigger matrix sets, for example, ∼2000 in Li et al [2013] and 1000 (synthetic) in Armstrong and Rendell [2008].…”

“…In existing work, features are usually determined according to the matrix storage formats, not code size. N and NZ are almost always collected as features (e.g., El Zein and Rendell [2012], Rendell [2008, 2010], Li et al [2013], and Neelima et al [2014] [Li et al 2013;Neelima et al 2014], and -memory traffic (number of bytes fetched, number of writes to w) [Belgin et al 2011].…”

“…Previous autotuning approaches for SpMV focus on choosing an optimal storage format, because even the basic sparsity regime of a matrix can have a profound effect on the performance [Bell and Garland 2009]. To this end, there exist work using decision trees [Li et al 2013], dynamic-programming [Guo et al 2014], reinforcement learning [Armstrong and Rendell 2008], heuristic-based autotuning [Abu-Sufah and Abdel Karim 2013], and model-driven approaches [Neelima et al 2014;Choi et al 2010]. To the best of our knowledge, ours is the first study on applying autotuning to pick among several specialization methods.…”

A New Approach for Sparse Matrix Classification Based on Deep Learning Techniques

“…We can find in the literature many analytical approaches that deal with the identification of the optimal sparse matrix format for GPUs based on performance models [12][13][14]. They show a good accuracy but models are usually tested considering a small set of matrices.…”

Kepler GPU accelerated recursive sorting using dynamic parallelism

“…Using shared memory, there can be substantial performance gains, when compared with the global memory access, which takes far more clock cycles. The readers are directed to some of the work carried out by the first author towards various optimizations and usage of GPU for scientific computations at .…”