Improving the Performance of the Symmetric Sparse Matrix-Vector Multiplication in Multicore

Gkountouvas, Theodoros; Karakasis, Vasileios; Kourtis, Kornilios; Goumas, Georgios; Koziris, Nectarios

doi:10.1109/ipdps.2013.43

Cited by 14 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The major challenge here is to resolve the potential write conflicts of explicit SymmSpMV kernels in parallel processing. There are general solutions for such problems like lock-based methods and thread private target arrays [10,17,27,36]. However, they have in common that their overhead may increase with the degree of parallelism.…”

Section: Introduction and Related Workmentioning

confidence: 99%

A Recursive Algebraic Coloring Technique for Hardware-efficient Symmetric Sparse Matrix-vector Multiplication

Alappat

Hager

Schenk

et al. 2020

ACM Trans. Parallel Comput.

View full text Add to dashboard Cite

The symmetric sparse matrix-vector multiplication (SymmSpMV) is an important building block for many numerical linear algebra kernel operations or graph traversal applications. Parallelizing SymmSpMV on today's multicore platforms with up to 100 cores is difficult due to the need to manage conflicting updates on the result vector. Coloring approaches can be used to solve this problem without data duplication, but existing coloring algorithms do not take load balancing and deep memory hierarchies into account, hampering scalability and full-chip performance. In this work, we propose the recursive algebraic coloring engine (RACE), a novel coloring algorithm and open-source library implementation that eliminates the shortcomings of previous coloring methods in terms of hardware efficiency and parallelization overhead. We describe the level construction, distance-k coloring, and load balancing steps in RACE, use it to parallelize SymmSpMV, and compare its performance on 31 sparse matrices with other state-of-the-art coloring techniques and Intel MKL on two modern multicore processors. RACE outperforms all other approaches substantially. By means The project is funded by the German DFG priority programme 1648 "Software for Exascale Computing (SPPEXA)" and the Swiss National Science Foundation (SNF) under the projects "Dual-Phase Steels-From Micro to Macro Properties (EXASTEEL-2)" (DFG, SNF) and "Equipping Sparse Solvers for Exascale (ESSEX-II)" (DFG).

show abstract

Section: Introduction and Related Workmentioning

confidence: 99%

A Recursive Algebraic Coloring Technique for Hardware-efficient Symmetric Sparse Matrix-vector Multiplication

Alappat

Hager

Schenk

et al. 2020

ACM Trans. Parallel Comput.

View full text Add to dashboard Cite

show abstract

“…Symmetric results. We note that considerable effort has been made for tuning hardware to efficiently handle "matrix-vector multiply": the multiplication of a vector by a sparse symmetric matrix (for example, see [11] and the references therein). Considering that virtually any use of a generalized inverse H would involve matrix-vector multiply, it can be very useful to prepare a sparse symmetric generalized inverse H from a symmetric A.…”

Section: Rankmentioning

confidence: 99%

Approximate 1-norm minimization and minimum-rank structured sparsity for various generalized inverses via local search

Xu¹,

Fampa²,

Lee³

et al. 2019

Preprint

View full text Add to dashboard Cite

Fundamental in matrix algebra and its applications, a generalized inverse of a real matrix A is a matrix H that satisfies the Moore-Penrose (M-P) property AHA = A. If H also satisfies the additional useful M-P property, HAH = H, it is called a reflexive generalized inverse. We consider aspects of symmetry related to the calculation of a sparse reflexive generalized inverse of A. As is common, and following Lee and Fampa (2018) for calculating sparse generalized inverses, we use (vector) 1-norm minimization for inducing sparsity and for keeping the magnitude of entries under control.When A is symmetric, we may naturally desire a symmetric H; while generally such a restriction on H may not lead to a 1-norm minimizing reflexive generalized inverse. Letting the rank of A be r, and seeking a 1-norm minimizing symmetric reflexive generalized inverse H, we give (i) a closed form when r = 1, (ii) a closed form when r = 2 and A is non-negative, and (iii) an approximation algorithm for general r. Importantly, our symmetric reflexive generalized inverse is structured and has guaranteed sparsity (≤ r 2 nonzeros).Other aspects of symmetry that we consider relate to the other two M-P properties: H is ahsymmetric if AH is symmetric, and ha-symmetric if HA is symmetric. Here we do not assume that A is symmetric, and we do not impose symmetry on H. Seeking a 1-norm minimizing ah-symmetric (or ha-symmetric) reflexive generalized inverse H, we give (i) a closed form when r = 1, (ii) a closed form when r = 2 and A satisfies a technical condition, and (iii) an approximation algorithm for general r. Importantly, our ah-symmetric (ha-symmetric) reflexive generalized inverse is structured and has better guaranteed sparsity (≤ mr nonzeros) than the 1-norm minimizing ah-symmetric (ha-symmetric) reflexive generalized inverse obtained via linear programming.

show abstract

“…In these simulations, the Sparse Matrix-Vector (S p MV) product plays a fundamental role for the iterative solution of sparse linear systems, as it frequently dictates the execution time and energy consumption of the whole algorithm. Indeed, the S p MV operation has been characterized as one of the most important computational kernels in science and engineering for the last decade (Gkountouvas et al, 2013).…”

Section: Introductionmentioning

confidence: 99%

Convolutional neural nets for estimating the run time and energy consumption of the sparse matrix-vector product

Barreda

Dolz

Castaño

2020

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

Modeling the performance and energy consumption of the sparse matrix-vector product (SpMV) is essential to perform off-line analysis and, for example, choose a target computer architecture that delivers the best performance-energy consumption ratio. However, this task is especially complex given the memory-bounded nature and irregular memory accesses of the SpMV, mainly dictated by the input sparse matrix. In this paper, we propose a Machine Learning (ML)-driven approach that leverages Convolutional Neural Networks (CNNs) to provide accurate estimations of the performance and energy consumption of the SpMV kernel. The proposed CNN-based models use a blockwise approach to make the CNN architecture independent of the matrix size. These models are trained to estimate execution time as well as total, package, and DRAM energy consumption at different processor frequencies. The experimental results reveal that the overall relative error ranges between 0.5% and 14%, while at matrix level is not superior to 10%. To demonstrate the applicability and accuracy of the SpMV CNN-based models, this study is complemented with an ad-hoc time-energy model for the PageRank algorithm, a popular algorithm for web information retrieval used by search engines, which internally realizes the SpMV kernel.

show abstract

Improving the Performance of the Symmetric Sparse Matrix-Vector Multiplication in Multicore

Cited by 14 publications

References 24 publications

A Recursive Algebraic Coloring Technique for Hardware-efficient Symmetric Sparse Matrix-vector Multiplication

A Recursive Algebraic Coloring Technique for Hardware-efficient Symmetric Sparse Matrix-vector Multiplication

Approximate 1-norm minimization and minimum-rank structured sparsity for various generalized inverses via local search

Convolutional neural nets for estimating the run time and energy consumption of the sparse matrix-vector product

Contact Info

Product

Resources

About