Speculative segmented sum for sparse matrix-vector multiplication on heterogeneous processors

Liu, Weifeng; Vinter, Brian

doi:10.1016/j.parco.2015.04.004

Cited by 54 publications

(35 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to SpGEMM, other sparse matrix multiplication operations (e.g., multiplication of sparse matrix and dense matrix [10,11,25] and its special case sparse matrix-vector multiplication [7,8,9,26,27]) pre-allocate a dense resulting matrix or vector. Thus the size of the result of the multiplication is trivially predictable, and the corresponding entries are stored to predictable memory addresses.…”

Section: Memory Pre-allocation For the Resulting Matrixmentioning

confidence: 99%

A framework for general sparse matrix–matrix multiplication on GPUs and heterogeneous processors

Liu

Vinter

2015

Journal of Parallel and Distributed Computing

Self Cite

View full text Add to dashboard Cite

h i g h l i g h t s• We design a framework for SpGEMM on modern manycore processors using the CSR format.• We present a hybrid method for pre-allocating the resulting sparse matrix.• We propose an efficient parallel insert method for long rows of the resulting matrix.• We develop a heuristic-based load balancing strategy. • Our approach significantly outperforms other known CPU and GPU SpGEMM methods. a b s t r a c tGeneral sparse matrix-matrix multiplication (SpGEMM) is a fundamental building block for numerous applications such as algebraic multigrid method (AMG), breadth first search and shortest path problem. Compared to other sparse BLAS routines, an efficient parallel SpGEMM implementation has to handle extra irregularity from three aspects: (1) the number of nonzero entries in the resulting sparse matrix is unknown in advance, (2) very expensive parallel insert operations at random positions in the resulting sparse matrix dominate the execution time, and (3) load balancing must account for sparse data in both input matrices.In this work we propose a framework for SpGEMM on GPUs and emerging CPU-GPU heterogeneous processors. This framework particularly focuses on the above three problems. Memory pre-allocation for the resulting matrix is organized by a hybrid method that saves a large amount of global memory space and efficiently utilizes the very limited on-chip scratchpad memory. Parallel insert operations of the nonzero entries are implemented through the GPU merge path algorithm that is experimentally found to be the fastest GPU merge approach. Load balancing builds on the number of necessary arithmetic operations on the nonzero entries and is guaranteed in all stages.Compared with the state-of-the-art CPU and GPU SpGEMM methods, our approach delivers excellent absolute performance and relative speedups on various benchmarks multiplying matrices with diverse sparsity structures. Furthermore, on heterogeneous processors, our SpGEMM approach achieves higher throughput by using re-allocatable shared virtual memory.

show abstract

Section: Memory Pre-allocation For the Resulting Matrixmentioning

confidence: 99%

A framework for general sparse matrix–matrix multiplication on GPUs and heterogeneous processors

Liu

Vinter

2015

Journal of Parallel and Distributed Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…The number of nonzero elements of the matrices ranges from 100K to 200M [21]. The dataset includes both regular and irregular matrices, covering domains from scientific computing to social networks [24]. 0 2 4 6 8 10 12 14 Speedups over NUMA-unaware(x) Fig.…”

Section: Evaluation Setupmentioning

confidence: 99%

Optimizing Sparse Matrix–Vector Multiplications on an ARMv8-based Many-Core Architecture

Chen

Fang

Chen

et al. 2019

Int J Parallel Prog

View full text Add to dashboard Cite

Sparse matrix-vector multiplications (SpMV) are common in scientific and HPC applications but are hard to be optimized. While the ARMv8based processor IP is emerging as an alternative to the traditional x64 HPC processor design, there is little study on SpMV performance on such new many-cores. To design efficient HPC software and hardware, we need to understand how well SpMV performs. This work develops a quantitative approach to characterize SpMV performance on a recent ARMv8-based many-core architecture, Phytium FT-2000 Plus (FTP). We perform extensive experiments involved over 9,500 distinct profiling runs on 956 sparse datasets and five mainstream sparse matrix storage formats, and compare FTP against the Intel Knights Landing many-core. We experimentally show that picking the optimal sparse matrix storage format and parameters is non-trivial as the correct decision requires expert knowledge of the input matrix and the hardware. We address the problem by proposing a machine learning based model that predicts the best storage format and parameters using input matrix features. The model automatically specializes to the many-core architectures we considered. The experimental results show that our approach achieves on average 93% of the best-available performance without incurring runtime profiling overhead.

show abstract

“…These work include designing new programming language constructs [26] and developing parallelization frameworks [5,7,29,30,35]. Some of these studies have explored parallelism in irregular programs [9,15,20,25,31], which share some similarities with the parallelization of FSM computations, given that FSMs essentially run on an irregular data structure (a graph). Quinones and others [28] use pre-computation for speculative threading, which shares the idea with speculative FSM parallelization in that both exploit some constraints of the computation to facilitate speculative execution.…”

Section: Related Workmentioning

confidence: 99%

Enabling scalability-sensitive speculative parallelization for FSM computations

Qiu

Zhao

et al. 2017

Proceedings of the International Conference on Supercomputing

View full text Add to dashboard Cite

Finite state machines (FSMs) are the backbone of many applications, but are difficult to parallelize due to their inherent dependencies. Speculative FSM parallelization has shown promise on multicore machines with up to eight cores. However, as hardware parallelism grows (e.g., Xeon Phi has up to 288 logical cores), a fundamental question raises: How does the speculative FSM parallelization scale as the number of cores increases? Without answering this question, existing methods for speculative FSM parallelization simply choose to use all available cores, which might not only waste computing resources, but also result in suboptimal performance. In this work, we conduct a systematic scalability analysis for speculative FSM parallelization. Unlike many other parallelizations which can be modeled by the classic Amdahl's law or its simple extensions, speculative FSM parallelization scales unconventionally due to the non-deterministic nature of speculation and the cost variations of misspeculation. To address these challenges, this work introduces a spectrum of scalability models that are customized to the properties of specific FSMs and the underlying architecture. The models, for the first time, precisely capture the scalability of speculative parallelization for different FSM computations, and clearly show the existence of a "sweet spot" in terms of the number of cores employed by the speculative FSM parallelization to achieve the optimal performance. To make the scalability models practical, we develop S3, a scalability-sensitive speculation framework for FSM parallelization. For any given FSM, S3 can automatically characterize its properties and analyze its scalability, hence guide speculative parallelization towards the optimal performance and more efficient use of computing resources. Evaluations on different FSMs and architectures confirm the accuracy of the proposed models and show that S3 achieves significant speedup (up to 5X) and energy savings (up to 77%). CCS CONCEPTS • Computing methodologies → Parallel algorithms; • Computer systems organization → Parallel architectures;

show abstract

Speculative segmented sum for sparse matrix-vector multiplication on heterogeneous processors

Cited by 54 publications

References 42 publications

A framework for general sparse matrix–matrix multiplication on GPUs and heterogeneous processors

A framework for general sparse matrix–matrix multiplication on GPUs and heterogeneous processors

Optimizing Sparse Matrix–Vector Multiplications on an ARMv8-based Many-Core Architecture

Enabling scalability-sensitive speculative parallelization for FSM computations

Contact Info

Product

Resources

About