Mixed parallel implementations of the top level step of Strassen and Winograd matrix multiplication algorithms

Abstract

This paper presents parallel implementations of the top level of Strassen and Winograd algorithms for matrix multiplication that use mixed-parallelism, i.e., simultaneous exploitation of data-and task-parallelism. This paradigm allows a better task placement and reduces the communication costs. A comparison with the ScaLAPACK implementation of the matrix multiplication is given. We present a theoretical evaluation of the algorithms which is corroborated by experiments.

Impact of mixed‐parallelism on parallel implementations of the Strassen and Winograd matrix multiplication algorithms

“…The first is inspired by the basic representation of the Strassen algorithm presented in Figure 1, as it follows the phases of this algorithm. The version given in this paper is an improvement of that presented in [9]. The second implementation has been designed as the result of a list scheduling algorithm.…”

“…Then phase 3 is only computed on one context as for Strassen. This choice unbalances the amount of computation between contexts, but reduces the amount of data exchanged with regard to the implementation presented in [9]. Both algorithms for contexts 1 and 2 are presented in Figure 9.…”

“…In [9], the authors proposed implementations of the Strassen and Winograd algorithms using mixedparallelism which need 2(M/p) 2 temporary elements on each processor of context 1 and 3(M/p) temporary elements on each processor of context 2. The implementations presented in this paper clearly reduce this amount.…”

“…This reduction comes from a change in the communication policy which has an influence on the number of temporary elements required. In [9], the authors performed message …”

Parallel implementation of Strassen's matrix multiplication algorithm for heterogeneous clusters

“…Matrix multiplication is one of the most important kernels for scientific applications, and Strassen's algorithm [11] is a matrix multiplication algorithm of complexity O(n log 2 7 ). Several implementations of parallel Strassen's algorithm which can compute matrix multiplications faster than standard O(n 3 ) algorithms [10,7,4], have been proposed.…”