Strassen’s Communication-Avoiding Parallel Matrix Multiplication Algorithm for All-Port 2D Torus Networks

Baransel, Cesur

doi:10.1007/978-3-642-39958-9_1

Cited by 1 publication

(2 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Optimizations of matrix multiplication have been extensively studied in distributed computing, scientific computing and high-performance computing [3]- [8]. In recent studies, it has been argued that communication costs are and will continue to be the limiting factor in the design of efficient parallel algorithms [9]. As a result, some significant communication-avoiding matrix multiplication algorithms have been proposed to address the bottleneck in HPC domain [3], [9]- [12].…”

Section: Introductionmentioning

confidence: 99%

“…In recent studies, it has been argued that communication costs are and will continue to be the limiting factor in the design of efficient parallel algorithms [9]. As a result, some significant communication-avoiding matrix multiplication algorithms have been proposed to address the bottleneck in HPC domain [3], [9]- [12]. The state-of-the-art one is the Communication-Avoiding Parallel Strassen (CAPS) algorithm which reportedly performs asymptotically better than any previous classical or Strassen-based parallel algorithm and obtains 24% to 184% speedups on Cray XT4 [10].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Cloud-Friendly Communication-Optimal Implementation for Strassen's Matrix Multiplication Algorithm

Zhou

2015

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYDue to its on-demand and pay-as-you-go properties, cloud computing has become an attractive alternative for HPC applications. However, communication-intensive applications with complex communication patterns still cannot be performed efficiently on cloud platforms, which are equipped with MapReduce technologies, such as Hadoop and Spark. In particular, one major obstacle is that MapReduce's simple programming model cannot explicitly manipulate data transfers between compute nodes. Another obstacle is cloud's relatively poor network performance compared with traditional HPC platforms. The traditional Strassen's algorithm of square matrix multiplication has a recursive and complex pattern on the HPC platform. Therefore, it cannot be directly applied to the cloud platform. In this paper, we demonstrate how to make Strassen's algorithm with complex communication patterns "cloud-friendly". By reorganizing Strassen's algorithm in an iterative pattern, we completely separate its computations and communications, making it fit to MapReduce programming model. By adopting a novel data/task parallel strategy, we solve Strassen's data dependency problems, making it well balanced. This is the first instance of Strassen's algorithm in MapReduce-style systems, which also matches Strassen's communication lower bound. Further experimental results show that it achieves a speedup ranging from 1.03× to 2.50× over the classical Θ(n 3 ) algorithm. We believe the principle can be applied to many other complex scientific applications.

show abstract