Communication Avoiding Gaussian elimination

Grigori, Laura; Demmel, James; Xiang, Hua

doi:10.1109/sc.2008.5214287

Cited by 35 publications

(60 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We describe it in Section 3.2. We also address the issue of designing algorithms recently referred to as communication reducing/minimizing/avoiding/optimal algorithms [3,9,15]. This is a difficult problem and is a subject of current research in the field of DLA.…”

Section: Design Philosophymentioning

confidence: 99%

“…A classic example is the transition from algorithms based on optimized Level 1 BLAS (from the LINPACK and EISPACK libraries) to algorithms that use block matrix operations in their innermost loops, which actually formed LAPACK's design philosophy. Current examples include work on LU and QR factorizations, in particular in the so called tiled [7] and communication avoiding [9,15] algorithms. We developed a hybrid LU algorithm, described in Section 4, that is yet another example of a communication-optimal algorithm.…”

Section: Design Philosophymentioning

confidence: 99%

“…Reference [15] describes a pivoting strategy that minimizes the number of messages exchanged during the panel factorization and demonstrates that this approach is stable in practice. For multicore, pairwise pivoting (PwP) is often considered (e.g., in [7]) but this generates a significant overhead since the rows are swapped in pairs of blocks.…”

Section: The Issue Of Pivoting In Lu Factorizationsmentioning

confidence: 99%

See 2 more Smart Citations

Towards dense linear algebra for hybrid GPU accelerated manycore systems

2010

View full text Add to dashboard Cite

a b s t r a c tWe highlight the trends leading to the increased appeal of using hybrid multicore + GPU systems for high performance computing. We present a set of techniques that can be used to develop efficient dense linear algebra algorithms for these systems. We illustrate the main ideas with the development of a hybrid LU factorization algorithm where we split the computation over a multicore and a graphics processor, and use particular techniques to reduce the amount of pivoting and communication between the hybrid components. This results in an efficient algorithm with balanced use of a multicore processor and a graphics processor.

show abstract

Section: Design Philosophymentioning

confidence: 99%

Section: Design Philosophymentioning

confidence: 99%

Section: The Issue Of Pivoting In Lu Factorizationsmentioning

confidence: 99%

See 1 more Smart Citation

Towards dense linear algebra for hybrid GPU accelerated manycore systems

2010

View full text Add to dashboard Cite

show abstract

“…To identify the set of b candidate pivot columns, a tournament is performed based on a reduction operation, where at each node of the reduction tree b candidate columns are selected by using the strong rank revealing QR factorization. The idea of tournament pivoting has been first used to reduce communication in Gaussian elimination [18,19], and then in the context of a newly introduced LU factorization with panel rank revealing pivoting [24]. CARRQR is optimal in terms of communication, modulo polylogarithmic factors, on both sequential machines with two levels of slow and fast memory and parallel machines with one level of parallelism, while performing three times more floating point operations than QRCP.…”

mentioning

confidence: 99%

“…Our communication avoiding algorithm, CARRQR, is based on tournament pivoting, and uses a reduction operation on blocks of columns to identify the next b pivot columns at each step of the block algorithm. This idea is analogous to the reduction operation used in CALU [18] to identify the next b pivot rows. The operator used at each node of the reduction tree is a RRQR factorization.…”

mentioning

confidence: 99%

Communication Avoiding Rank Revealing QR Factorization with Column Pivoting

Demmel¹,

Grigori²,

Gu³

et al. 2013

Self Cite

View full text Add to dashboard Cite

Abstract. In this paper we introduce CARRQR, a communication avoiding rank revealing QR factorization with tournament pivoting. We show that CARRQR reveals the numerical rank of a matrix in an analogous way to QR factorization with column pivoting (QRCP). Although the upper bound of a quantity involved in the characterization of a rank revealing factorization is worse for CARRQR than for QRCP, our numerical experiments on a set of challenging matrices show that this upper bound is very pessimistic, and CARRQR is an effective tool in revealing the rank in practical problems.Our main motivation for introducing CARRQR is that it minimizes data transfer, modulo polylogarithmic factors, on both sequential and parallel machines, while previous factorizations as QRCP are communication sub-optimal and require asymptotically more communication than CARRQR. Hence CARRQR is expected to have a better performance on current and future computers, where commmunication is a major bottleneck that highly impacts the performance of an algorithm.

show abstract

Parallel spherical harmonic transforms on heterogeneous architectures (graphics processing units/multi‐core CPUs)

Szydlarski

Esterie

Falcou

et al. 2013

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

SUMMARY Spherical harmonic transforms (SHT) are at the heart of many scientific and practical applications ranging from climate modelling to cosmological observations. In many of these areas, new cutting‐edge science goals have been recently proposed requiring simulations and analyses of experimental or observational data at very high resolutions and of unprecedented volumes. Both these aspects pose formidable challenge for the currently existing implementations of the transforms. This paper describes parallel algorithms for computing SHT with two variants of intra‐node parallelism appropriate for novel supercomputer architectures, multi‐core processors and Graphic Processing Units (GPU). It also discusses their performance, alone and embedded within a top‐level, Message Passing Interface‐based parallelisation layer ported from the S2HAT library, in terms of their accuracy, overall efficiency and scalability. We show that our inverse SHT run on GeForce 400 Series GPUs equipped with latest Compute Unified Device Architecture architecture (Fermi) outperforms the state of the art implementation for a multi‐core processor executed on a current Intel Core i7‐2600K. Furthermore, we show that an Message Passing Interface/Compute Unified Device Architecture version of the inverse transform run on a cluster of 128 Nvidia Tesla S1070 is as much as 3 times faster than the hybrid Message Passing Interface/OpenMP version executed on the same number of quad‐core processors Intel Nehalem for problem sizes motivated by our target applications. Performance of the direct transforms is however found to be at the best comparable in these cases. We discuss in detail the algorithmic solutions devised for the major steps involved in the transforms calculation, emphasising those with a major impact on their overall performance and elucidates the sources of the dichotomy between the direct and the inverse operations.Copyright © 2013 John Wiley & Sons, Ltd.

show abstract

Communication Avoiding Gaussian elimination

Cited by 35 publications

References 7 publications

Towards dense linear algebra for hybrid GPU accelerated manycore systems

Towards dense linear algebra for hybrid GPU accelerated manycore systems

Communication Avoiding Rank Revealing QR Factorization with Column Pivoting

Parallel spherical harmonic transforms on heterogeneous architectures (graphics processing units/multi‐core CPUs)

Contact Info

Product

Resources

About