Mumps

Ghoting, Amol; Gunnels, John A.; Squillante, Mark S.; Meseguer, José; Cownie, James H.; Roweth, Duncan; Adve, Sarita V.; Boehm, Hans-J.; McKee, Sally A.; Wisniewski, Robert W.; Karypis, George; Malony, Allen D.; Gottlieb, Steven; Riesen, Rolf; Maccabe, Arthur B.; Bilardi, Gianfranco; Pietracaprina, Andrea; Kejariwal, Arun; Nicolau, Alexandru; Lengauer, Christian; Gustafson, John L.; Gropp, William; Prost, Jean-Pierre; Padua, David; Lowney, Geoff; Halstead, Robert H.; Nemirovsky, Mario; Amestoy, Patrick; Buttari, Alfredo; Duff, Iain; Guermouche, Abdou; L’Excellent, Jean-Yves; Uçar, Bora; Pakin, Scott

doi:10.1007/978-0-387-09766-4_204

Cited by 7 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ey also require minimal performance tuning to extract performance on various GPU generations. ese new kernels represent critical building blocks for designing e cient sparse direct solvers on hardware accelerators [Amestoy et al 2011;Hénon et al 2002] as well as in the context of machine learning applications with low-rank matrix approximations [Akbudak et al 2017]. Future work includes extending these kernels to support non-uniform matrix sizes and a lightweight auto-tuning framework to optimize these batched operations transparently.…”

Section: Discussionmentioning

confidence: 99%

Batched Triangular Dense Linear Algebra Kernels for Very Small Matrix Sizes on GPUs

Charara

Keyes

Ltaief

2019

ACM Trans. Math. Softw.

View full text Add to dashboard Cite

Batched dense linear algebra kernels are becoming ubiquitous in scienti c applications, ranging from tensor contractions in deep learning to data compression in hierarchical low-rank matrix approximation. Within a single API call, these kernels are capable of simultaneously launching up to thousands of similar matrix computations, removing the expensive overhead of multiple API calls while increasing the occupancy of the underlying hardware. A challenge is that for the existing hardware landscape (x86, GPUs, etc.), only a subset of the required batched operations is implemented by the vendors, with limited support for very small problem sizes. We describe the design and performance of a new class of batched triangular dense linear algebra kernels on very small data sizes (up to 256) using single and multiple GPUs. By deploying two-sided recursive formulations, stressing the register usage, maintaining data locality, reducing threads synchronization and fusing successive kernel calls, the new batched kernels outperform existing state-of-the-art implementations.A. Charara et al.However, some of the most critical applications currently of high interests in the HPC community, especially in data analytics, face a major performance bo leneck due to the inadequacy of legacy BLAS/LAPACK frameworks. For instance, tensor contractions for deep learning and hierarchical low rank data-sparse matrix computations [Hackbusch 1999;Hackbusch and Khoromskij 2000] are key operations for solving partial di erential equations. Interestingly, the bulk of the computation of these operations typically resides in performing thousands of independent dense linear algebra operations on very small sizes (usually less than 100). Even the highly vendor-optimized sequential implementations may not cope with the overhead of the memory latency at these tiny sizes. Moreover, calling the sequential version of the dense linear algebra functions within an embarrassingly parallel OpenMP loop may not be an option, due to the API overhead (i.e., parameters sanity check, memory initialization, etc.), which does not get compensated in return because of the low arithmetic intensity of the kernel operations. is is further exacerbated by hardware with a large number of threads, such as GPUs with many streaming multiprocessors, for which high occupancy may not be reached, bandwidth may not get saturated, and thread parallelism may not be exploited given the small workloads. At present, vendors currently provide only a subset of the overall batched linear algebra operations, with limited support for very small problem sizes.is paper describes the high-performance implementations on GPUs of various batched triangular dense linear algebra operations targeting very small sizes (up to 256 in dimension), which are currently either poorly supported or not at all. ere are two main algorithmic adaptations, which may address this challenge: designing synchronization-reducing (i.e., strong scaling) and communication-reducing (i.e., data motion avoiding) algorithms. Alth...

show abstract

Section: Discussionmentioning

confidence: 99%

Batched Triangular Dense Linear Algebra Kernels for Very Small Matrix Sizes on GPUs

Charara

Keyes

Ltaief

2019

ACM Trans. Math. Softw.

View full text Add to dashboard Cite

show abstract

“…In [29], a reordering strategy is proposed for the multifrontal solver Mumps [2]. The objective is to provide a row ordering and the associated mapping on a set of processors to minimize the total volume of communications.…”

Section: Related Workmentioning

confidence: 99%

“…Let's now define for each vertex the binary vector B i = (row ik ) k∈ 1, −1 . We can then define w i , the weight of a row i, as in (2), which represents the number of supernodes contributing to that row i, and the distance between two rows i and j, d i,j , as in (3). This is known as the Hamming distance [13] between two binary vectors and allows for measuring the number of off-diagonal blocks induced by the succession of two rows i and j.…”

Section: Problem Modelingmentioning

confidence: 99%

Reordering Strategy for Blocking Optimization in Sparse Linear Solvers

Pichon¹,

Faverge²,

Ramet³

et al. 2017

SIAM J. Matrix Anal. & Appl.

View full text Add to dashboard Cite

Solving sparse linear systems is a problem that arises in many scientific applications, and sparse direct solvers are a time-consuming and key kernel for those applications and for more advanced solvers such as hybrid direct-iterative solvers. For this reason, optimizing their performance on modern architectures is critical. The preprocessing steps of sparse direct solvers-ordering and block-symbolic factorization-are two major steps that lead to a reduced amount of computation and memory and to a better task granularity to reach a good level of performance when using BLAS kernels. With the advent of GPUs, the granularity of the block computation has become more important than ever. In this paper, we present a reordering strategy that increases this block granularity. This strategy relies on block-symbolic factorization to refine the ordering produced by tools such as Metis or Scotch, but it does not impact the number of operations required to solve the problem. We integrate this algorithm in the PaStiX solver and show an important reduction of the number of off-diagonal blocks on a large spectrum of matrices. This improvement leads to an increase in efficiency of up to 20% on GPUs.

show abstract