Combining building blocks for parallel multi-level matrix multiplication

Hunold, Sascha; Rauber, Thomas; Rünger, Gudula

doi:10.1016/j.parco.2008.03.003

Cited by 14 publications

(13 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other parallel approaches [17,22,31] have used more complex parallel schemes and communication patterns. However, they restrict attention to only one or two steps of Strassen and obtain modest performance improvements over classical algorithms.…”

Section: Previous Work On Parallel Strassenmentioning

confidence: 99%

Communication-optimal parallel algorithm for strassen's matrix multiplication

Ballard

Demmel

Holtz

et al. 2012

Proceedings of the Twenty-Fourth Annual ACM Symposium on Parallelism in Algorithms and Architectures

113

154

View full text Add to dashboard Cite

Parallel matrix multiplication is one of the most studied fundamental problems in distributed and high performance computing. We obtain a new parallel algorithm that is based on Strassen's fast matrix multiplication and minimizes communication. The algorithm outperforms all known parallel matrix multiplication algorithms, classical and Strassen-based, both asymptotically and in practice.A critical bottleneck in parallelizing Strassen's algorithm is the communication between the processors. Ballard, Demmel, Holtz, and Schwartz (SPAA'11) prove lower bounds on these communication costs, using expansion properties of the underlying computation graph. Our algorithm matches these lower bounds, and so is communication-optimal. It exhibits perfect strong scaling within the maximum possible range. * Research supported by Microsoft (Award #024263) and Intel (Award #024894) funding and by matching funding by U.C. Discovery (Award #DIG07-10227 Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SPAA'12, June 25-27, 2012, Pittsburgh, Pennsylvania, USA. Copyright 2012 ACM 978-1-4503-0743-7/11/06 ...$10.00.Benchmarking our implementation on a Cray XT4, we obtain speedups over classical and Strassen-based algorithms ranging from 24% to 184% for a fixed matrix dimension n = 94080, where the number of nodes ranges from 49 to 7203.Our parallelization approach generalizes to other fast matrix multiplication algorithms.

show abstract

Section: Previous Work On Parallel Strassenmentioning

confidence: 99%

Communication-optimal parallel algorithm for strassen's matrix multiplication

Ballard

Demmel

Holtz

et al. 2012

Proceedings of the Twenty-Fourth Annual ACM Symposium on Parallelism in Algorithms and Architectures

113

154

View full text Add to dashboard Cite

show abstract

“…Second, by the parallel implementation [13,75,42,29,19,47]. Of course, a combination of these two approaches is also possible (see, for instance, [72,37,32,49]). …”

mentioning

confidence: 99%

“…extra levels of latency of data access). However, with a lot of programmer's work, hybrid approaches outperformed the standard parallel matrix multiplication [72,37,32,59]. Interestingly, the most recent research seems to contain two contradictory claims.…”

mentioning

confidence: 99%

Generalized Matrix Multiplication and its Object Oriented Model

Ganzha¹,

Paprzycki²,

Sedukhin³

2014

SCPE

View full text Add to dashboard Cite

Abstract.Since the beginning of the 21st century, we observe rapid changes in the area of, broadly understood, computational sciences. One of interesting effects of these changes is the need for reevaluation of the role of dense matrix multiplication. The aim of this paper is two-fold. First, to summarize developments that point toward a need for reconsidering usefulness of matrix multiplication generalized on the basis of the theory of algebraic semirings. Second, to propose generalized matrix-matrix multiply-and-update (MMU) operation and its object oriented model.Key words: matrix multiplication, algebraic semirings, algebraic path problem AMS subject classifications. 65F30, 13A991. Introduction. Recently, a number of changes can be observed in computational sciences. They concern all levels of the computational stack. First, evolution of computer hardware, forced by limits imposed by physics, resulted in practical disappearance of processors with a single computational unit. As a matter of fact, today it is possible to have a quad-core processor in a cell phone (e.g. in the newest Samsung Galaxy 4) and even 8 cores (in the Motorola X8 Mobile Computing System [10]). Furthermore, it is already possible to have more than a thousand fused multiply and add (FMA) units in a single GPU processor [33]. Second, there is a constantly growing gap between the capacity of the processor to consume the data and hardware' ability to feed it. Third, rapidly decreasing cost of the FMA unit, combined with appearance of processors with thousands FMAs, lead to suggestions that a complete reevaluation of approach to computing is needed [34,35]. Here, the basic assumption is that data access/movement is "expensive," while arithmetic operations are "cheap." Fourth, it is time to (re)consider complexity of codes that try to match (and effectively utilize) current computer hardware with as much as seven levels of data access latency. Finally, rapid proliferation of devices with matrix-like sensor input (e.g. digital cameras, medical imaging devices, radio telescopes, etc.) forcefully reminds us that, in multiple applications, actual data consists of 2D and/or 3D matrix-structures that are fed with high speed, and should not be stored but processed in-place as they elements are delivered to the processing units.In this paper we will argue that the time has come for a meta-reflection and general change of approach to large-scale (primarily "scientific") computing. In particular, it is important to look into efficient solution of matrix-based problems, and this is precisely the scope of the current contribution. This paper modifies and extends our two conference papers [69,30], and it is organized as follows. First, we discuss the interaction between progress in computer hardware and computational linear algebra in the early days of supercomputing. Second, we consider dense matrix multiplication, as one of key elements of large number of linear algebraic algorithms. Here, we also look into its generalization through the theory of algebr...

show abstract

“…Recently, the matrix multiplication computational complexity has been reduced using Strassens algorithm [5]. In [5], author replace the number of matrix Figure 1.…”

mentioning

confidence: 99%

“…Recently, the matrix multiplication computational complexity has been reduced using Strassens algorithm [5]. In [5], author replace the number of matrix Figure 1. Matrix multiplication by Strassen's algorithm [4] multiplication by matrix additions that reduced the time complexity of matrix multiplication from O(n 3 ) to O(n 2 .8).…”

mentioning

confidence: 99%

Task Partitioning and Load Balancing Strategy for Matrix Applications on Distributed System

Bashir¹,

Madani²,

Kazmi³

et al. 2013

JCP

View full text Add to dashboard Cite

In this paper, we present a load-balancing strategy (Adaptive Load Balancing strategy) for data parallel applications to balance the work load effectively on a distributed system. We study its impact on computation-hungry matrix multiplication application. The ALB strategy enhances the performance with features such as intelligent node selection, pre-task assignment, adaptive task sizing and buffer allocation, and load balancing. The ALB strategy exhibits reduced nodes idle time and inter process communication time, and improved speed up as compared to Run Time task Scheduling strategy.

show abstract

Combining building blocks for parallel multi-level matrix multiplication

Cited by 14 publications

References 13 publications

Communication-optimal parallel algorithm for strassen's matrix multiplication

Communication-optimal parallel algorithm for strassen's matrix multiplication

Generalized Matrix Multiplication and its Object Oriented Model

Task Partitioning and Load Balancing Strategy for Matrix Applications on Distributed System

Contact Info

Product

Resources

About