Reducing Parallel Communication in Algebraic Multigrid through Sparsification

Abstract

Algebraic multigrid (AMG) is an O(n) solution process for many large sparse linear systems. A hierarchy of progressively coarser grids is constructed that utilize complementary relaxation and interpolation operators. High-energy error is reduced by relaxation, while low-energy error is mapped to coarse-grids and reduced there. However, large parallel communication costs often limit parallel scalability. As the multigrid hierarchy is formed, each coarse matrix is formed through a triple matrix product. The resulting coarse-grids often have significantly more nonzeros per row than the original fine-grid operator, thereby generating high parallel communication costs on coarse-levels. In this paper, we introduce a method that systematically removes entries in coarse-grid matrices after the hierarchy is formed, leading to an improved communication costs. We sparsify by removing weakly connected or unimportant entries in the matrix, leading to improved solve time. The main trade-off is that if the heuristic identifying unimportant entries is used too aggressively, then AMG convergence can suffer. To counteract this, the original hierarchy is retained, allowing entries to be reintroduced into the solver hierarchy if convergence is too slow. This enables a balance between communication cost and convergence, as necessary. In this paper we present new algorithms for reducing communication and present a number of computational experiments in support.

FFT, FMM, and multigrid on the road to exascale: Performance challenges and opportunities

“…) cache misses, for each transferring line of size L [13,14]. This bound is optimal, matching the lower bound by Hong and Kung [15] when 3 √ N is an exact power of two. Thus, the time spent moving data between the main memory and a processor in the 3-D FFT is given by…”

“…Generally, the FFT is used for uniform discretizations, FMM and geometric MG are efficient solvers on irregular grids with local features or discontinuities, and algebraic MG can handle arbitrary geometries, variable coefficients, and general boundary conditions. The focus of this study is on FFT, FMM, and geometric MG, although several observations extend to an algebraic setting as well [3].…”

Nonsymmetric Algebraic Multigrid Based on Local Approximate Ideal Restriction ($\ell$AIR)

“…For nonsymmetric matrices, we can note that in relaxing only on A f f , error propagation and residual propagation are similar in the F-F block: I − ∆A f f = ∆(I − A f f ∆)∆ −1 . However, the connection between error and residual reduction in general is less clear when considering C-points and F-points, as in (5) and (6). Residual reduction is based on the column scaling of A and error reduction on the row-scaling.…”

“…were eliminated in row i that were smaller than 0.001 · max j |a ij |. It is known that such an approach is typically not effective on diffusive matrices, prompting research into more advanced techniques for reducing the number of matrix nonzeros [6,17,54]. Here, we use a technique similar to the elimination used in [33], but instead of actually eliminating entries, we add them to the diagonal in order to preserve the row sum.…”

Scaling Structured Multigrid to 500K+ Cores Through Coarse-Grid Redistribution

“…The efficient solution of large, sparse linear systems resulting from the discretization of elliptic partial differential equations (PDEs) is crucial to the performance of many numerical simulations. Although there has been significant progress in developing general Algebraic Multigrid (AMG) solvers [27], modern highperformance computing (HPC) architectures continue to pose significant challenges to parallel scalability and performance (e.g., [3,13,5]). These challenges include reducing data movement, increasing arithmetic intensity, and identifying opportunities to improve resilience, and are more readily addressed in settings where problem structure can be identified and exploited.…”