Parallel Bidiagonal SVD via the Method of Multiple Relatively Robust Representations

Abstract

A bidiagonal SVD solver based on the Method of Multiple Relatively Robust Representations (MR 3 ) promises to be an improvement over the current state of the art by providing lower asymptotic runtime and the ability to calculate a subset of singular triplets at reduced cost. Currently, no MR 3 based SVD solver is readily available. Using results by Willems and Lang [2], we provide an initial implementation of such a solver and compare it to LAPACK's Divide and Conquer solver. The Method of Multiple Relatively Robust RepresentationsRoughly speaking, MR 3 is similar to Inverse Iteration with the extra property that the computed eigenvectors do not require explicit orthogonalization. To achieve orthogonality of the eigenvectors, MR 3 treats clusters-eigenvalues with small relative distances-separately. If τ is just outside a cluster, then the shift T → T − Iτ has the effect of enlarging the relative distances of the clustered eigenvalues, so that the respective eigenvectors can be computed while maintaining orthogonality. To ensure accuracy, the shifted matrices are stored in robust representations, usually variants of LDL * decompositions. Unlike the Divide and Conquer method, MR 3 does not make any use of high level BLAS; therefore, in order to take advantage of multithreading, an explicit parallelization is required. In light of results by Petschow and Bientinesi [1], we use a task based parallelization approach which has the potential of yielding high parallel scalability. MR 3 as an SVD solverObtaining an accurate SVD solver from MR 3 is not a trivial matter. Willems and Lang [2] published a method based on the Golub-Kahan form T GK (B) of a bidiagonal matrix B; for details see Equation 8.6.3 in [3]. The Golub-Kahan form has the property that (σ, u, v) is a singular triplet of B if, and only if, (±σ, q ± ) is an eigenpair of T GK (B). Moreover, a singular vector of B can cheaply be obtained from the corresponding eigenvector q ± . Note, that two eigenvalues, −σ and +σ of T GK (B), correspond to one singular value σ of B. The entire SVD of B can be obtained from half the spectrum of the Golub-Kahan form, by calculating either all the negative or all the positive eigenvalues.The application of a black box MR 3 to the Golub-Kahan form does not yield an accurate solver. There are two problems which are related to loss of the Golub-Kahan structure. First, for robustness reasons, MR 3 starts out by shifting the input matrix to be definite. As a consequence, all eigenvalues of small magnitude become part of the same cluster. The problem is that the subspaces spanned by the eigenvectors corresponding to +σ and −σ can not be cleanly separated. As a result, the eigenvectors of small eigenvalues lose the Golub-Kahan structure and the extracted singular vectors are then not orthonormal.The other problem is that with repeated shifts, the LDL * representations may become increasingly dissimilar to the matrix that the decomposition is meant to represent. Such a phenomenon is caused by element growth: the decomposition uses ...