Enhancing Parallelism of Tile Bidiagonal Transformation on Multicore Architectures Using Tree Reduction

Abstract

Abstract. The objective of this paper is to enhance the parallelism of the tile bidiagonal transformation using tree reduction on multicore architectures. First introduced by Ltaief et. al [LAPACK Working Note #247, 2011], the bidiagonal transformation using tile algorithms with a two-stage approach has shown very promising results on square matrices. However, for tall and skinny matrices, the inherent problem of processing the panel in a domino-like fashion generates unnecessary sequential tasks. By using tree reduction, the panel is horizontally split, which creates another dimension of parallelism and engenders many concurrent tasks to be dynamically scheduled on the available cores. The results reported in this paper are very encouraging. The new tile bidiagonal transformation, targeting tall and skinny matrices, outperforms the state-of-the-art numerical linear algebra libraries LAPACK V3.2 and Intel MKL ver. 10.3 by up to 29-fold speedup and the standard two-stage PLASMA BRD by up to 20-fold speedup, on an eight socket hexa-core AMD Opteron multicore shared-memory system.

Energy Footprint of Advanced Dense Numerical Linear Algebra Using Tile Algorithms on Multicore Architectures

“…Mixed precisions techniques are also used for further approximating contributions from farther particles in the context of fast multipole methods as well as LQCD computation [13], resulting in speeding up the computation while reducing memory traffic. Moreover, tree reduction techniques were naturally identified as a way to increase concurrency while reducing data motion on multicore architecture [14]- [16]. In this work, we present detailed power analysis of these mixed precision and tree reduction codes.…”

An improved parallel singular value algorithm and its implementation for multicore hardware

“…The two-stage approach was applied to the TRD (Triangular Reduction) [34] and to SVD [35,53,54] in combination with tile algorithms and runtime scheduling based on data dependences between tasks that operate on the tiles. This resulted in very good performance but has never been used to compute the singular vectors.…”

“…The caveat is that the reductions can be done easily to a band form, instead of the proper bi-diagonal matrix or a tri-diagonal matrix (with a single subdiagonal). The solution is to reduce to the band form first, and then produce the proper form through the process of bulge chasing, i.e., successive elimination of the subdiagonal entries by a series of Householder transformations [34,35,[52][53][54][55]. Because both the reduction to the band form and the bulge chasing process can be implemented in a parallel and cache-efficient manner, the two-stage procedure is an order of magnitude faster than the legacy approach of LAPACK, which relies heavily on Level 2 BLAS operations, is memory bound, and therefore inefficient.…”

“…In order to exploit the fine-grained parallelism to its fullest, efficient schedules have to be designed, while data dependencies are preserved, i.e., data hazards are prevented. This has been done for both the simpler single-sided factorizations, such as Cholesky, LU and QR [1, 2, 13, 14, 19-21, 36, 48], as well as the more complicated two-sided factorizations, such as the reductions to band bi-diagonal and band tri-diagonal form [34,35,[52][53][54][55]. The process of constructing such schedules through manipulation of loop indexes and enforcing them by progress tables is tedious and error-prone.…”

“…The QR iteration [17,18] is no longer a method of choice for singular vectors because it takes roughly 50% longer than the methods mentioned earlier. The optimization techniques we present here are only applicable to sqaure matrices because a slightly different approach is necessariy for the rectangular ones [54].…”