Communication-Avoiding Krylov Techniques on Graphic Processing Units

MehriDehnavi, Maryam; El-Kurdi, Yousef; Demmel, James; Giannacopoulos, Dennis D.

doi:10.1109/tmag.2013.2244861

Cited by 10 publications

(7 citation statements)

References 12 publications

(12 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Related Workmentioning

confidence: 99%

“…A number of libraries and inspector-executor frameworks provide parallel implementations of fused sparse kernels with no loop-carried dependencies such as, two or more SpMV kernels [2,21,31,34,40] or SpMV and dot products [1,2,13,15,40,61]. The formulation of 𝑠-step Krylov solvers [6] has enabled iterations of iterative solvers to be interleaved and hence multiple SpMV kernels are optimized simultaneously via replicating computations to minimize communication costs [21,31,34,45]. Sparse tiling [26,[47][48][49]51] is an inspector executor approach that uses manually written inspectors [47,49] to group iteration of different loops of a specific kernel such as Gauss-Seidel [49] and Moldyn [47] and is generalized for parallel loops without loop-carried dependencies [26,51].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Composing Loop-carried Dependence with Other Loops

Cheshmi¹,

Strout²,

Dehnavi³

2021

Preprint

View full text Add to dashboard Cite

Sparse fusion is a compile-time loop transformation and runtime scheduling implemented as a domain-specific code generator. Sparse fusion generates efficient parallel code for the combination of two sparse matrix kernels where at least one of the kernels has loop-carried dependencies. Available implementations optimize individual sparse kernels. When optimized separately, the irregular dependence patterns of sparse kernels create synchronization overheads and load imbalance, and their irregular memory access patterns result in inefficient cache usage, which reduces parallel efficiency. Sparse fusion uses a novel inspection strategy with code transformations to generate parallel fused code for sparse kernel combinations that is optimized for data locality and load balance. Code generated by Sparse fusion outperforms the existing implementations ParSy and MKL on average 1.6× and 5.1× respectively and outperforms the LBC and DAGP coarsening strategies applied to a fused data dependence graph on average 5.1× and 7.2× respectively for various kernel combinations.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Composing Loop-carried Dependence with Other Loops

Cheshmi¹,

Strout²,

Dehnavi³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Most of these work focus on accelerating the execution of the compute intensive kernels in the Krylov solvers. As shown in [4,56] such efforts are mainly communication bound and are limited by the maximum performance achieved from parallelizing the compute intensive kernels. The assembled matrix is usually large and sparse and in many cases does not fit in the small and fast access memory levels of CPU and the co-processor memory.…”

Section: Introductionmentioning

confidence: 99%

Parallel finite element technique using Gaussian belief propagation

El-Kurdi

Dehnavi

Gross

et al. 2015

Computer Physics Communications

Self Cite

View full text Add to dashboard Cite

a b s t r a c tThe computational efficiency of Finite Element Methods (FEMs) on parallel architectures is severely limited by conventional sparse iterative solvers. Conventional solvers are based on a sequence of global algebraic operations that limits their parallel efficiency. Traditionally, sophisticated programming techniques tailored to specific CPU architectures are used to improve the poor performance of sparse algebraic kernels. The introduced FEM Multigrid Gaussian Belief Propagation (FMGaBP) algorithm is a novel technique that eliminates all global algebraic operations and sparse data-structures. The algorithm is based on reformulating the FEM into a distributed variational inference problem on graphical models. We present new formulations for FMGaBP, which enhance its computation and communication complexities. A Helmholtz problem is used to validate the FMGaBP formulation for 2D, 3D and higher FEM degrees. Implementation techniques for multicore architectures that exploit the parallel features of FMGaBP are presented showing speedups compared to open-source libraries, specifically deal.II and Trilinos. FMGaBP is also implemented on manycore architectures in this work; Speedups of 4.8X, 2.3X and 1.5X are achieved on an NVIDIA Tesla C2075 compared to the parallel CPU implementation of FMGaBP on dual-core, quad-core and 12-core CPUs respectively.

show abstract

“…This motivated research into the use of other more well-conditioned bases for the Krylov subspaces, including scaled monomial bases [26], Chebyshev bases [29,16,17], and Newton bases [1,20]. 3 The growing cost of communication in large-scale sparse problems has created a recent resurgence of interest in the implementation, optimization, and development of s-step Krylov subspace methods; see, e.g., the recent works [18,27,35,46,23,34,44,45,43].…”

mentioning

confidence: 99%

“…The growing cost of communication in large-scale sparse problems has created a recent resurgence of interest in the implementation, optimization, and development of s-step Krylov subspace methods; see, e.g., the recent works [18,27,35,46,23,34,44,45,43].…”

mentioning

confidence: 99%

The Adaptive $s$-Step Conjugate Gradient Method

Carson¹

2018

SIAM J. Matrix Anal. & Appl.

View full text Add to dashboard Cite

On modern large-scale parallel computers, the performance of Krylov subspace iterative methods is limited by global synchronization. This has inspired the development of s-step (or communication-avoiding) Krylov subspace method variants, in which iterations are computed in blocks of s. This reformulation can reduce the number of global synchronizations per iteration by a factor of O(s), and has been shown to produce speedups in practical settings.Although the s-step variants are mathematically equivalent to their classical counterparts, they can behave quite differently in finite precision depending on the parameter s. If s is chosen too large, the s-step method can suffer a convergence delay and a decrease in attainable accuracy relative to the classical method. This makes it difficult for a potential user of such methods -the s value that minimizes the time per iteration may not be the best s for minimizing the overall time-to-solution, and further may cause an unacceptable decrease in accuracy.Towards improving the reliability and usability of s-step Krylov subspace methods, in this work we derive the adaptive s-step CG method, a variable s-step CG method where in block k, the parameter s k is determined automatically such that a user-specified accuracy is attainable. The method for determining s k is based on a bound on growth of the residual gap within block k, from which we derive a constraint on the condition numbers of the computed O(s k )-dimensional Krylov subspace bases. The computations required for determining the block size s k can be performed without increasing the number of global synchronizations per block. Our numerical experiments demonstrate that the adaptive s-step CG method is able to attain up to the same accuracy as classical CG while still significantly reducing the total number of global synchronizations.

show abstract

Communication-Avoiding Krylov Techniques on Graphic Processing Units

Cited by 10 publications

References 12 publications

Composing Loop-carried Dependence with Other Loops

Composing Loop-carried Dependence with Other Loops

Parallel finite element technique using Gaussian belief propagation

The Adaptive $s$-Step Conjugate Gradient Method

Contact Info

Product

Resources

About