Hypergraph-partitioning-based decomposition for parallel sparse-matrix vector multiplication

Abstract

AbstractÐIn this work, we show that the standard graph-partitioning-based decomposition of sparse matrices does not reflect the actual communication volume requirement for parallel matrix-vector multiplication. We propose two computational hypergraph models which avoid this crucial deficiency of the graph model. The proposed models reduce the decomposition problem to the well-known hypergraph partitioning problem. The recently proposed successful multilevel framework is exploited to develop a multilevel hypergraph partitioning tool PaToH for the experimental verification of our proposed hypergraph models. Experimental results on a wide range of realistic sparse test matrices confirm the validity of the proposed hypergraph models. In the decomposition of the test matrices, the hypergraph models using PaToH and hMeTiS result in up to 63 percent less communication volume (30 to 38 percent less on the average) than the graph model using MeTiS, while PaToH is only 1.3±2.3 times slower than MeTiS on the average. Index TermsÐSparse matrices, matrix multiplication, parallel processing, matrix decomposition, computational graph model, graph partitioning, computational hypergraph model, hypergraph partitioning.

Parallel image restoration using surrogate constraint methods

“…First, the objective in hypergraph-based formulation is an exact measure of the total communication volume. Second, the matrices in our methods are unsymmetric; standard graph partitioning is not readily applicable [4,13,14].…”

“…We use the column-net hypergraph model of Çatalyürek and Aykanat [4] to obtain a K-way rowwise partition on matrix H. The rowwise partition on H induces a columnwise partition on H T . As is known [13,34], the communication requirements of the row-parallel y ← H x and the column-parallel q ← H T multiplies are the same in terms of the total volume and number of messages.…”

“…Sparse matrix partitioning for parallel matrix-vector multiplies of the form y ← H x is formulated in terms of the graph [17] and hypergraph partitioning problems [4]. Both of these problems are NP-complete [10,23].…”

“…Both one-dimensional (1D), e.g., rowwise, and two-dimensional (2D), e.g., checkerboard, sparse matrix partitioning techniques are investigated (Sections 4 and 5, respectively). The recently proposed hypergraph partitioning models [3,4,6,34] are used for load balancing and communication overhead minimization for both partitioning frameworks. Parallel algorithms for BSCM and PSCM are implemented for a message-passing multicomputer based on the above mentioned partitioning frameworks.…”

Multi-level direct K-way hypergraph partitioning with multiple constraints and fixed vertices

“…In the literature, combinatorial models based on hypergraph partitioning are proposed for various complex and irregular problems arising in parallel scientific computing [4,10,17,26,50,53], VLSI design [2,42], software engineering [6], and database design [22,23,41,43,46]. These models formulate an original problem as a hypergraph partitioning problem, trying to optimize a certain objective function (e.g., minimizing the total volume of communication in parallel volume rendering, optimizing the placement of circuitry on a dice area, minimizing the access to disk pages in processing GIS queries) while maintaining a constraint (e.g., balancing the computational load in a parallel system, using disk page capacities as an upper bound in data allocation) imposed by the problem.…”