SURAA: A Novel Method and Tool for Loadbalanced and Coalesced SpMV Computations on GPUs

Abstract

Sparse matrix-vector (SpMV) multiplication is a vital building block for numerous scientific and engineering applications. This paper proposes SURAA (translates to speed in arabic), a novel method for SpMV computations on graphics processing units (GPUs). The novelty lies in the way we group matrix rows into different segments, and adaptively schedule various segments to different types of kernels. The sparse matrix data structure is created by sorting the rows of the matrix on the basis of the nonzero elements per row ( n p r) and forming segments of equal size (containing approximately an equal number of nonzero elements per row) using the Freedman–Diaconis rule. The segments are assembled into three groups based on the mean n p r of the segments. For each group, we use multiple kernels to execute the group segments on different streams. Hence, the number of threads to execute each segment is adaptively chosen. Dynamic Parallelism available in Nvidia GPUs is utilized to execute the group containing segments with the largest mean n p r, providing improved load balancing and coalesced memory access, and hence more efficient SpMV computations on GPUs. Therefore, SURAA minimizes the adverse effects of the n p r variance by uniformly distributing the load using equal sized segments. We implement the SURAA method as a tool and compare its performance with the de facto best commercial (cuSPARSE) and open source (CUSP, MAGMA) tools using widely used benchmarks comprising 26 high n p r v a r i a n c e matrices from 13 diverse domains. SURAA outperforms the other tools by delivering 13.99x speedup on average. We believe that our approach provides a fundamental shift in addressing SpMV related challenges on GPUs including coalesced memory access, thread divergence, and load balancing, and is set to open new avenues for further improving SpMV performance in the future.

Sehaa: A Big Data Analytics Tool for Healthcare Symptoms and Diseases Detection Using Twitter, Apache Spark, and Machine Learning

“…A range of technologies is contributing to the development of these smart systems. These include the Internet of Things (IoT) [32][33][34][35][36], social media [21][22][23]37,38], big data [39][40][41][42][43][44], high performance computing (HPC) [45][46][47][48], cloud, fog, and edge computing [34,[49][50][51][52], and machine learning [36,53]. The applications include healthcare [34,39,[54][55][56], transportation [57,58], and others [59,60].…”

Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading

“…The sparsity of a matrix and a suitable data structure determine the computational cost as well as the transmission time [31]. Adaptive partitioning here employs the Compressed Sparse Row (CSR) storage scheme [32]. The size of a matrix in CSR format is 12 •ẑ + 4 • (m + 1) bytes, whereẑ is the total number of non-zero elements and m the number of rows; in contrast, a dense matrix requires 8 • m • n bytes.…”

ZAKI+: A Machine Learning Based Process Mapping Tool for SpMV Computations on Distributed Memory Architectures

“…High performance computing (HPC) typically exploits parallel computing features of the underlying software and hardware infrastructure to solve large problems faster. HPC has been applied to SpMV/linear algebra [39]- [42], and other problems for several decades. Big data and data-driven approaches [35], [36], [43], [44] have been used relatively recently in scientific computing to address HPC related challenges, and this has given rise to the convergence of HPC and big data [45], [46].…”

“…The characteristics of the hardware platforms that could affect the SpMV performance include the DRAM bandwidth, cache hierarchy, the available parallelism in the hardware, and others. A range of hardware architecture are being used for SpMV implementations including CPUs [55]- [57], MIC [40], GPUs [39], [58], and other architectures [59], [60].…”

“…Modern machines have a range of shared and distributed memory, and hybrid architectures, with several hierarchies involving non-uniform communication latencies [61], [62]. The sparsity pattern of the matrix affects the performance of SpMV computations, particularly in the case of distributed-memory implementations (due to higher variations in communication delays), resulting in load imbalance that causes both computation and communication overheads [39]. The goal of mapping of sparse matrices and vectors to processors in distributed memory parallel environment is to minimize the overall communication (number of sent messages, communication volume per processor, synchronization costs) and provide computation load balance.…”