Model-driven autotuning of sparse matrix-vector multiply on GPUs

Abstract

We present a performance model-driven framework for automated performance tuning (autotuning) of sparse matrix-vector multiply (SpMV) on systems accelerated by graphics processing units (GPU). Our study consists of two parts.First, we describe several carefully hand-tuned SpMV implementations for GPUs, identifying key GPU-specific performance limitations, enhancements, and tuning opportunities. These implementations, which include variants on classical blocked compressed sparse row (BCSR) and blocked ELLPACK (BELLPACK) storage formats, match or exceed state-of-the-art implementations. For instance, our best BELLPACK implementation achieves up to 29.0 Gflop/s in single-precision and 15.7 Gflop/s in doubleprecision on the NVIDIA T10P multiprocessor (C1060), enhancing prior state-of-the-art unblocked implementations (Bell and Garland, 2009) by up to 1.8× and 1.5× for single-and doubleprecision respectively.However, achieving this level of performance requires input matrix-dependent parameter tuning. Thus, in the second part of this study, we develop a performance model that can guide tuning. Like prior autotuning models for CPUs (e.g., Im, Yelick, and Vuduc, 2004), this model requires offline measurements and run-time estimation, but more directly models the structure of multithreaded vector processors like GPUs. We show that our model can identify the implementations that achieve within 15% of those found through exhaustive search.

Accurate CUDA performance modeling for sparse matrix-vector multiplication

“…Choi et al [12] designed a blocked ELLPACK format and proposed a CUDA performance model to predict matrixdependent tuning parameters. Xu et al [13] proposed the optimized SpMV based on ELL format and a SpMV CUDA performance model.…”

Speculative segmented sum for sparse matrix-vector multiplication on heterogeneous processors

“…Because of less off-chip memory access and better on-chip memory localization, block-based formats or libraries, such as OSKI [38,42,43], pOSKI [37], CSB [44,45], BELLPACK [46], BCCOO/BCCOO+ [5], BRC [6] and RSB [47], attracted the most attention. However, block-based formats heavily rely on sparsity structure, meaning that the input matrix is required to have a block structure to meet potential block layout.…”

A Hybrid Circular Queue Method for Iterative Stencil Computations on GPUs

“…Their method is general and requires that global memory bandwidth is not the bottleneck for performance. In [35], a new compressed format is proposed for sparse matrix on GPU, and search is needed to determine certain parameters of the format. They proposed an analytical model specific to SpMV to eliminate search candidates.…”