Recursion based parallelization of exact dense linear algebra routines for Gaussian elimination

Abstract

We present block algorithms and their implementation for the parallelization of sub-cubic Gaussian elimination on shared memory architectures. Contrarily to the classical cubic algorithms in parallel numerical linear algebra, we focus here on recursive algorithms and coarse grain parallelization. Indeed, sub-cubic matrix arithmetic can only be achieved through recursive algorithms making coarse grain block algorithms perform more efficiently than fine grain ones. This work is motivated by the design and implementation of dense linear algebra over a finite field, where fast matrix multiplication is used extensively and where costly modular reductions also advocate for coarse grain block decomposition. We incrementally build efficient kernels, for matrix multiplication first, then triangular system solving, on top of which a recursive PLUQ decomposition algorithm is built. We study the parallelization of these kernels using several algorithmic variants: either iterative or recursive and using different splitting strategies. Experiments show that recursive adaptive methods for matrix multiplication, hybrid recursive-iterative methods for triangular system solve and tile recursive versions of the PLUQ decomposition, together with various data mapping policies, provide the best performance on a 32 cores NUMA architecture. Overall, we show that the overhead of modular reductions is more than compensated by the fast linear algebra algorithms and that exact dense linear algebra matches the performance of full rank reference numerical software even in the presence of rank deficiencies.

The Parallel Tiled WZ Factorization Algorithm for Multicore Architectures

“…The issues concerning parallel numerical algorithms and particularly Gaussian elimination on multicore architectures were presented, among others, by Dumas et al (2016) and Buttari et al (2009). Dumas et al (2016) present investigations into the parallelization of sub-cubic Gaussian elimination on shared memory multicore architectures. They focus on the parallelization of three sub-routines, namely, matrix multiplication, triangular system solving and the PLUQ factorization.…”

An Experimental Evaluation of the OpenMP Thread Mapping for LU Factorisation on Xeon Phi Coprocessor and on Hybrid CPU-MIC Platform

“…Recently, several groups have been working on the efficient parallel linear algebra libraries, particularly the Gaussian elimination. The Gaussian elimination on multicore and manycore architectures was studied, among others, in works [9], [2], [6] and [8]. In the work [9] the authors investigated the parallelization of sub-cubic Gaussian elimination.…”

“…The Gaussian elimination on multicore and manycore architectures was studied, among others, in works [9], [2], [6] and [8]. In the work [9] the authors investigated the parallelization of sub-cubic Gaussian elimination. They focused on the parallelization of three subroutines, namely, the matrix multiplication, the triangular equation solver and the LU factorisation with pivoting.…”

Fast computation of the rank profile matrix and the generalized Bruhat decomposition

“…This reduces the data locality of the algorithm, and therefore penalizes the efficiency of implementations in practice. In parallel, this blocking also puts more constrains on the dependencies between tasks Dumas et al (2015a).…”