Simultaneous Input and Output Matrix Partitioning for Outer-Product--Parallel Sparse Matrix-Matrix Multiplication

Abstract

Abstract. For outer-product-parallel sparse matrix-matrix multiplication (SpGEMM) of the form C = A×B, we propose three hypergraph models that achieve simultaneous partitioning of input and output matrices without any replication of input data. All three hypergraph models perform conformable one-dimensional (1D) columnwise and 1D rowwise partitioning of the input matrices A and B, respectively. The first hypergraph model performs two-dimensional (2D) nonzero-based partitioning of the output matrix, whereas the second and third models perform 1D rowwise and 1D columnwise partitioning of the output matrix, respectively. This partitioning scheme induces a two-phase parallel SpGEMM algorithm, where communication-free local SpGEMM computations constitute the first phase and the multiple single-node-accumulation operations on the local SpGEMM results constitute the second phase. In these models, the two partitioning constraints defined on weights of vertices encode balancing computational loads of processors during the two separate phases of the parallel SpGEMM algorithm. The partitioning objective of minimizing the cutsize defined over the cut nets encodes minimizing the total volume of communication that will occur during the second phase of the parallel SpGEMM algorithm. An MPI-based parallel SpGEMM library is developed to verify the validity of our models in practice. Parallel runs of the library for a wide range of realistic SpGEMM instances on two large-scale parallel systems JUQUEEN (an IBM BlueGene/Q system) and SuperMUC (an Intel-based cluster) show that the proposed hypergraph models attain high speedup values. 1. Introduction. Sparse matrix-matrix multiplication (SpGEMM) is a kernel operation in a wide variety of scientific applications such as finite element simulations based on domain decomposition [3,22], molecular dynamics (MD) [15,16,17,25,28,29,32,36], and linear programming (LP) [7,8,26], all of which utilize parallel processing technology to reduce execution times. Among these applications, below we exemplify three methods/codes from which we select realistic SpGEMM instances.In finite element application fields, finite element tearing and interconnecting (FETI) [3,22] type domain decomposition methods are used for numerical solution of engineering problems. In this application, the SpGEMM computation GG T is performed, where G = R T B T , R is the block diagonal basis of the stiffness matrix, and B is the signed matrix with entries −1, 0, 1 describing the subdomain interconnectivity.In MD application fields, CP2K program [1] performs parallel atomistic and

Hypergraph Partitioning for Parallel Sparse Matrix-Matrix Multiplication

“…The model proposed in Definition 1 is also distinct from the ones proposed in [1]. The approach in [1] considers a restricted class of parallelizations, known as outer-product algorithms, which leads to hypergraphs with fewer vertices and nets, while our model encompasses 1D (which include outer-product), 2D, and 3D parallelizations as defined in [2].…”

“…The approach in [1] considers a restricted class of parallelizations, known as outer-product algorithms, which leads to hypergraphs with fewer vertices and nets, while our model encompasses 1D (which include outer-product), 2D, and 3D parallelizations as defined in [2]. Another difference in the models is that the approach in [1] simultaneously partitions scalar multiplications and output matrix data, while we partition only the multiplications, as described above. Proof.…”

“…More recently, hypergraph models have been proposed for the outer-product algorithm for SpGEMM [1]. In this work, we present a hypergraph model that generalizes the "fine-grain" SpMV model [7] and considers a more general class of SpGEMM algorithms than [1] to minimize communication.…”

“…More recently, hypergraph models have been proposed for the outer-product algorithm for SpGEMM [1]. In this work, we present a hypergraph model that generalizes the "fine-grain" SpMV model [7] and considers a more general class of SpGEMM algorithms than [1] to minimize communication. Hypergraphs have been used to model more general computations than SpGEMM [10]: we differ Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage, and that copies bear this notice and the full citation on the first page.…”

“…The main contributions of this work are (1) showing that SpGEMM can be modeled by a hypergraph so that its communication cost corresponds to the size of a cut induced by a partition of the vertices and (2) demonstrating that hypergraph partitioners can be used to determine efficient SpGEMM algorithms for an algebraic multigrid application.…”