OpenMP Issues Arising in the Development of Parallel BLAS and LAPACK Libraries

Ren, Yuhe; Waveren, Matthijs van

doi:10.1155/2003/278167

Cited by 11 publications

(19 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• The pipelined algorithm from [1] does not perform nearly as well as the data-flow algorithms proposed in this paper.…”

Section: Resultsmentioning

confidence: 78%

“…The problem with this idea is that it greatly complicates the code that implements the algorithm [1,20,24]. While doable for a single, relatively simple algorithm like the Cholesky factorization, reimplementation of an entirely library like LAPACK becomes a daunting task when these techniques are employed.…”

Section: Multithreaded Blasmentioning

confidence: 99%

“…Pipelined algorithm (Serial BLAS) Our implementation of the first algorithm in [1] which includes compute-ahead and pipelining. We made a best-effort attempt to incorporate optimizations similar to those for other implementations.…”

Section: Implementationsmentioning

confidence: 99%

See 2 more Smart Citations

Supermatrix out-of-order scheduling of matrix operations for SMP and multi-core architectures

Chan

Quintana–Ort́ı

Quintana-Ortí

et al. 2007

Proceedings of the Nineteenth Annual ACM Symposium on Parallel Algorithms and Architectures

100

View full text Add to dashboard Cite

We discuss the high-performance parallel implementation and execution of dense linear algebra matrix operations on SMP architectures, with an eye towards multi-core processors with many cores. We argue that traditional implementations, as those incorporated in LAPACK, cannot be easily modified to render high performance as well as scalability on these architectures. The solution we propose is to arrange the data structures and algorithms so that matrix blocks become the fundamental units of data, and operations on these blocks become the fundamental units of computation, resulting in algorithms-byblocks as opposed to the more traditional blocked algorithms. We show that this facilitates the adoption of techniques akin to dynamic scheduling and out-of-order execution usual in superscalar processors, which we name SuperMatrix Out-of-Order scheduling. Performance results on a 16 CPU Itanium2-based server are used to highlight opportunities and issues related to this new approach.

show abstract

“…• The pipelined algorithm from [1] does not perform nearly as well as the data-flow algorithms proposed in this paper.…”

Section: Resultsmentioning

confidence: 78%

Section: Multithreaded Blasmentioning

confidence: 99%

See 1 more Smart Citation

Supermatrix out-of-order scheduling of matrix operations for SMP and multi-core architectures

Chan

Quintana–Ort́ı

Quintana-Ortí

et al. 2007

Proceedings of the Nineteenth Annual ACM Symposium on Parallel Algorithms and Architectures

100

View full text Add to dashboard Cite

show abstract

“…The problem with this idea is that it greatly complicates the code that implements the algorithm if coded in a traditional style [Addison et al 2003;Kurzak and Dongarra 2006;Strazdins 2001]. While feasible for a single, relatively simple algorithm like the LU factorization without pivoting or the Cholesky factorization, re-implementing a linear algebra library like LAPACK would become a daunting task if this strategy were employed.…”

Section: The Trouble With Evolving Legacy Code To Multithreaded Archimentioning

confidence: 99%

Programming matrix algorithms-by-blocks for thread-level parallelism

Quintana-Ortí

Quintana–Ort́ı

Geijn

et al. 2009

ACM Trans. Math. Softw.

116

128

View full text Add to dashboard Cite

With the emergence of thread-level parallelism as the primary means for continued performance improvement, the programmability issue has reemerged as an obstacle to the use of architectural advances. We argue that evolving legacy libraries for dense and banded linear algebra is not a viable solution due to constraints imposed by early design decisions. We propose a philosophy of abstraction and separation of concerns that provides a promising solution in this problem domain. The first abstraction, FLASH, allows algorithms to express computation with matrices consisting of contiguous blocks, facilitating algorithms-by-blocks. Operand descriptions are registered for a particular operation a priori by the library implementor. A runtime system, SuperMatrix, uses this information to identify data dependencies between suboperations, allowing them to be scheduled to threads out-of-order and executed in parallel. But not all classical algorithms in linear algebra lend themselves to conversion to algorithms-by-blocks. We show how our recently proposed LU factorization with incremental pivoting and a closely related algorithm-by-blocks for the QR factorization, both originally designed for out-of-core computation, overcome this difficulty. Anecdotal evidence regarding the development of routines with a core functionality demonstrates how the methodology supports high productivity while experimental results suggest that high performance is abundantly achievable.

show abstract

“…Jay Hoeflinger, Prasad Allavilli, Thomas Jackson and Bob Kuhn have studied scalability issues using OpenMP for CFD applications [9]. OpenMP issues in the development of parallel BLAS and LAPACK libraries have also been studied [2]. However, the issues, challenges related with programming and effective exploitation of shared memory multi-core systems with respect to cache parameters have not been considered.…”

Section: Related Workmentioning

confidence: 99%

Architecture Aware Programming on Multi-Core Systems

Pimple¹,

Sathe²

2011

IJACSA

View full text Add to dashboard Cite

Abstract-In order to improve the processor performance, the response of the industry has been to increase the number of cores on the die. One salient feature of multi-core architectures is that they have a varying degree of sharing of caches at different levels. With the advent of multi-core architectures, we are facing the problem that is new to parallel computing, namely, the management of hierarchical caches. Data locality features need to be considered in order to reduce the variance in the performance for different data sizes. In this paper, we propose a programming approach for the algorithms running on shared memory multi-core systems by using blocking, which is a wellknown optimization technique coupled with parallel programming paradigm, OpenMP. We have chosen the sizes of various problems based on the architectural parameters of the system like cache level, cache size, cache line size. We studied the cache optimization scheme on commonly used linear algebra applications -matrix multiplication (MM), Gauss-Elimination (GE) and LU Decomposition (LUD) algorithm.

show abstract

OpenMP Issues Arising in the Development of Parallel BLAS and LAPACK Libraries

Cited by 11 publications

References 11 publications

Supermatrix out-of-order scheduling of matrix operations for SMP and multi-core architectures

Supermatrix out-of-order scheduling of matrix operations for SMP and multi-core architectures

Programming matrix algorithms-by-blocks for thread-level parallelism

Architecture Aware Programming on Multi-Core Systems

Contact Info

Product

Resources

About