A GPU Framework for Sparse Matrix Vector Multiplication

Abstract

The hardware and software evolutions related to Graphics Processing Units (GPUs), for general purpose computations, have changed the way the parallel programming issues are addressed. Many applications are being ported onto GPU for achieving performance gain. The GPU execution time is continuously optimized by the GPU programmers while optimizing pre-GPU computation overheads attracted the research community in the recent past. While GPU executes the programs given by a CPU, pre-GPU computation overheads does exists and should be optimized for a better usage of GPUs. The GPU framework proposed in this paper improves the overall performance of the application by optimizing pre-GPU computation overheads along with GPU execution time. This paper proposes a sparse matrix format prediction tool to predict an optimal sparse matrix format to be used for a given input matrix by analyzing the input sparse matrix and considering pre-GPU computation overheads. The sparse matrix format predicted by the proposed method is compared against the best performing sparse matrix formats posted in the literature. The proposed model is based on the static data that is available from the input directly and hence the prediction overhead is very small. Compared to GPU specific sparse format prediction, the proposed model is more inclusive and precious in terms of increasing overall application's performance.

Accelerating the Fast Hadamard Single-Pixel Imaging (FHSI) on GPUs: Techniques and Optimizations

“…We leveraged the GPU-accelerated capabilities of the Nvidia Xavier platform, employing PyCUDA [22,23] to develop kernels aimed at optimizing bottleneck operations. Our optimization strategies included the use of shared memory with padding to prevent bank conflicts [24], coalescing global memory accesses for increased throughput [25], pre-computation of constants to diminish runtime calculations [26], loop unrolling [27], and warp divergence minimization through conditional optimization [4]. We also selected faster arithmetic operations when the precision requirements permitted and minimized synchronization needs.…”

Mapping and Optimization Method of SpMV on Multi-DSP Accelerator

“…Therefore, scholars have proposed some tools, such as SpMV Auto-Tuner (SMAT), to select the optimal compression format adapted to the hardware structure from various compression formats [9][10][11][12] by analyzing the distribution characteristics of non-zero elements in the sparse matrix. In addition, some researches have proposed to improve the performance of SpMV by accelerating the computing speed of the processor, such as using the heterogeneous parallel computing structure of CPU+GPU [13]. However, when the SpMV algorithm is running, the memory access to the compressed sparse matrix elements are continuous at the same time as the memory access to the x elements are irregular.…”

Kepler GPU accelerated recursive sorting using dynamic parallelism

“…Using shared memory, there can be substantial performance gains, when compared with the global memory access, which takes far more clock cycles. The readers are directed to some of the work carried out by the first author towards various optimizations and usage of GPU for scientific computations at .…”