Virtual Systolic Array for QR Decomposition

Kurzak, Jakub; Łuszczek, Piotr; Gates, Mark; Yamazaki, Ichitaro; Dongarra, Jack

doi:10.1109/ipdps.2013.119

Cited by 6 publications

(19 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare all the previous algorithms that were implemented with PaRSEC with the following algorithms from the literature [22] on the very same hardware:…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Implementing a Systolic Algorithm for QR Factorization on Multicore Clusters with PaRSEC

Aupy

Faverge

Robert

et al. 2014

Euro-Par 2013: Parallel Processing Workshops

Self Cite

View full text Add to dashboard Cite

This article introduces a new systolic algorithm for QR factorization, and its implementation on a supercomputing cluster of multicore nodes. The algorithm targets a virtual 3D-array and requires only local communications. The implementation of the algorithm uses threads at the node level, and MPI for internode communications. The complexity of the implementation is addressed with the PaRSEC software, which takes as input a parametrized dependence graph, which is derived from the algorithm, and only requires the user to decide, at the high-level, the allocation of tasks to nodes. We show that the new algorithm exhibits competitive performance with state-of-the-art QR routines on a supercomputer called Kraken, which shows that high-level programming environments, such as PaRSEC, provide a viable alternative to enhance the production of quality software on complex and hierarchical architectures.

show abstract

“…We compare all the previous algorithms that were implemented with PaRSEC with the following algorithms from the literature [22] on the very same hardware:…”

Section: Methodsmentioning

confidence: 99%

“…-SYSTOLIC-1D is the virtual systolic array decomposition [22]. As its name indicates, it targets a 1D-linear array of processors.…”

Section: Methodsmentioning

confidence: 99%

Implementing a Systolic Algorithm for QR Factorization on Multicore Clusters with PaRSEC

Aupy

Faverge

Robert

et al. 2014

Euro-Par 2013: Parallel Processing Workshops

Self Cite

View full text Add to dashboard Cite

show abstract

“…On such a systolic array, the execution of an operation is message-driven or data-stream-driven by data counters and triggered by the arrival of data objects. Though the hardware implementations of such systolic arrays had been haunted by a number of problems, the systolic array becames very attractive as a parallel programming model on modern computers when it is implemented as a software layer like the Virtual Systolic Array (VSA) that we presented earlier [4]. Since this discovery, the concepts of 1D, 2D, and 3D systolic arrays as virtualized software designs have been combined with a distributed-memory dataflow runtime and delivered a wide range of scalability results, but with varying levels of achievable performance [5].…”

Section: Introductionmentioning

confidence: 99%

“…Our previous work [4] presented a 2D systolic array for a tile dense QR decomposition algorithm with its panel factorization based on a flat-tree reduction. Its strong scaling performance for square matrices was superior to that of the state-of-the-art software, and could potentially tackle the challenges of utilizing the increasing levels of concurrency on the emerging supercomputers.…”

Section: Introductionmentioning

confidence: 99%

Design and Implementation of a Large Scale Tree-Based QR Decomposition Using a 3D Virtual Systolic Array and a Lightweight Runtime

Yamazaki

Kurzak

Łuszczek

et al. 2014

Parallel Process. Lett.

Self Cite

View full text Add to dashboard Cite

A systolic array provides an alternative computing paradigm to the von Neumann architecture. Though its hardware implementation has failed as a paradigm to design integrated circuits in the past, we are now discovering that the systolic array as a software virtualization layer can lead to an extremely scalable execution paradigm. To demonstrate this scalability, in this paper, we design and implement a 3D virtual systolic array to compute a tile QR decomposition of a tall-and-skinny dense matrix. Our implementation is based on a state-of-the-art algorithm that factorizes a panel based on a tree-reduction. Freed from the constraint of a planar layout, we present a three-dimensional virtual systolic array architecture for this algorithm. Using a runtime developed as a part of the Parallel Ultra Light Systolic Array Runtime (PULSAR) project, we demonstrate on a Cray-XT5 machine how our virtual systolic array can be mapped to a large-scale machine and obtain excellent parallel performance. This is an

show abstract

“…His assumption is that in the hardware of the future, cost of arithmetic operations will be so low, in comparison with data movement (and indexing), that fundamental assumptions behind current approaches to computational sparse (and dense) linear algebra will have to be re-evaluated. Interestingly, the team of J. Dongarra has recently initiated a new project, in which they investigate possibility of reintroduction of systolic architectures to the computing mainstream [43]. Here, it is also worthy envisioning that this could mean that we may, some time in the future, forget about rectangular matrices (in computational practice).…”

mentioning

confidence: 99%

Generalized Matrix Multiplication and its Object Oriented Model

Ganzha¹,

Paprzycki²,

Sedukhin³

2014

SCPE

View full text Add to dashboard Cite

Abstract.Since the beginning of the 21st century, we observe rapid changes in the area of, broadly understood, computational sciences. One of interesting effects of these changes is the need for reevaluation of the role of dense matrix multiplication. The aim of this paper is two-fold. First, to summarize developments that point toward a need for reconsidering usefulness of matrix multiplication generalized on the basis of the theory of algebraic semirings. Second, to propose generalized matrix-matrix multiply-and-update (MMU) operation and its object oriented model.Key words: matrix multiplication, algebraic semirings, algebraic path problem AMS subject classifications. 65F30, 13A991. Introduction. Recently, a number of changes can be observed in computational sciences. They concern all levels of the computational stack. First, evolution of computer hardware, forced by limits imposed by physics, resulted in practical disappearance of processors with a single computational unit. As a matter of fact, today it is possible to have a quad-core processor in a cell phone (e.g. in the newest Samsung Galaxy 4) and even 8 cores (in the Motorola X8 Mobile Computing System [10]). Furthermore, it is already possible to have more than a thousand fused multiply and add (FMA) units in a single GPU processor [33]. Second, there is a constantly growing gap between the capacity of the processor to consume the data and hardware' ability to feed it. Third, rapidly decreasing cost of the FMA unit, combined with appearance of processors with thousands FMAs, lead to suggestions that a complete reevaluation of approach to computing is needed [34,35]. Here, the basic assumption is that data access/movement is "expensive," while arithmetic operations are "cheap." Fourth, it is time to (re)consider complexity of codes that try to match (and effectively utilize) current computer hardware with as much as seven levels of data access latency. Finally, rapid proliferation of devices with matrix-like sensor input (e.g. digital cameras, medical imaging devices, radio telescopes, etc.) forcefully reminds us that, in multiple applications, actual data consists of 2D and/or 3D matrix-structures that are fed with high speed, and should not be stored but processed in-place as they elements are delivered to the processing units.In this paper we will argue that the time has come for a meta-reflection and general change of approach to large-scale (primarily "scientific") computing. In particular, it is important to look into efficient solution of matrix-based problems, and this is precisely the scope of the current contribution. This paper modifies and extends our two conference papers [69,30], and it is organized as follows. First, we discuss the interaction between progress in computer hardware and computational linear algebra in the early days of supercomputing. Second, we consider dense matrix multiplication, as one of key elements of large number of linear algebraic algorithms. Here, we also look into its generalization through the theory of algebr...

show abstract

Virtual Systolic Array for QR Decomposition

Cited by 6 publications

References 20 publications

Implementing a Systolic Algorithm for QR Factorization on Multicore Clusters with PaRSEC

Implementing a Systolic Algorithm for QR Factorization on Multicore Clusters with PaRSEC

Design and Implementation of a Large Scale Tree-Based QR Decomposition Using a 3D Virtual Systolic Array and a Lightweight Runtime

Generalized Matrix Multiplication and its Object Oriented Model

Contact Info

Product

Resources

About