Performance optimization, modeling and analysis of sparse matrix-matrix products on multi-core and many-core processors

Abstract

Sparse matrix-matrix multiplication (SpGEMM) is a computational primitive that is widely used in areas ranging from traditional numerical applications to recent big data analysis and machine learning. Although many SpGEMM algorithms have been proposed, hardware specific optimizations for multi-and many-core processors are lacking and a detailed analysis of their performance under various use cases and matrices is not available. We firstly identify and mitigate multiple bottlenecks with memory management and thread scheduling on Intel Xeon Phi (Knights Landing or KNL). Specifically targeting many-core processors, we develop a hashtable-based algorithm and optimize a heap-based shared-memory SpGEMM algorithm. We examine their performance together with other publicly available codes. Different from the literature, our evaluation also includes use cases that are representative of real graph algorithms, such as multi-source breadth-first search or triangle counting. Our hash-table and heap-based algorithms are showing significant speedups from libraries in the majority of the cases while different algorithms dominate the other scenarios with different matrix size, sparsity, compression factor and operation type. We wrap up in-depth evaluation results and make a recipe to give the best SpGEMM algorithm for target scenario. We build the performance model for hash-table and heap-based algorithms, which supports the recipe. A critical finding is that hash-table-based SpGEMM gets a significant performance boost if the nonzeros are not required to be sorted within each row of the output matrix. Finally, we integrate our implementations into a large-scale protein clustering code named HipMCL, accelerating its SpGEMM kernel by up to 10X and achieving an overall performance boost for the whole HipMCL application by 2.6X.

EXAGRAPH: Graph and combinatorial methods for enabling exascale applications

“…HipMCL is an iterative algorithm that relies on SpGEMM as its workhorse at each iteration. The ExaGraph project ported the hash-based SpGEMM algorithm, which was originally developed for GPUs by collaborators (Nagasaka et al, 2017), into multicore CPUs and Intel KNLs (Nagasaka et al, 2019). For GPU-equipped clusters, we developed a model to choose the fastest GPU-based SpGEMM depending on the sparsity of the current MCL iteration and utilized a pipelined communication scheme that hides the cost of CPU-to-GPU data transfers.…”

The parallelism motifs of genomic data analysis

“…The high-performance distributed re-implementation of the Markov Cluster algorithm, known as HipMCL [40], uses some of the most general and scalable sparse matrix algorithms implemented within the Combinatorial BLAS [41]. These algorithms include a two-dimensional SpGEMM algorithm known as Sparse SUMMA [24], several different shared memory SpGEMM algorithms [42] that are optimized for different iterations of HipMCL, a fast memory estimator based on sparse matrix dense matrix multiplication for memory-efficient SpGEMM [43], as well as a very fast distributed memory connected components algorithm [44] that is used for extracting the final clusters from the result of the HipMCL iterations. The integration of GPU support as well as faster communication-avoiding SpGEMM algorithms [45] is ongoing work.…”

“…These methods do not simply partition the result matrix or particles/sequences over processors, but instead replicate them to the extent allowed by available memory. For sparse matrices and sparse interactions, the benefits depend more on the sparsity patterns [24,42,43,70], but are useful in clustering [40] and possibly alignment.…”

BELLA: Berkeley Efficient Long-Read to Long-Read Aligner and Overlapper

“…The implementation of this method within our pipeline enables the use of high-performance techniques previously not applied in the context of long-read alignment. It also allows continuing performance improvements in this step due to the ever-improving optimized implementations of SpGEMM (Nagasaka et al, 2019;Deveci et al, 2017).…”

“…More importantly, the computational problem of accumulating the contributions from multiple shared k-mers to each pair of reads is handled automatically by the choice of appropriate data structures within SpGEMM. Figure 2 illustrates the merging operation of BELLA, which uses a hash table data structure indexed by the row indexes of A, following the multi-threaded implementation proposed by Nagasaka et al (2019). Finally, the contents of the hash table are stored into a column of the final matrix once all required nonzeros for that column are accumulated.…”