A Framework for the $MR^3$ Algorithm: Theory and Implementation

Willems, Paul R.; Lang, Bruno

doi:10.1137/110834020

Cited by 10 publications

(23 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[51,52,53,54] For the purposes of this paper, we note that ELPA relies upon an efficient implementation of the divide-and-conquer approach, the steps of which are reviewed in Ref. [22].…”

Section: The Eigenvalue Problemmentioning

confidence: 99%

The ELPA library: scalable parallel eigenvalue solutions for electronic structure theory and computational science

Marek

Blüm

Johanni

et al. 2014

J. Phys.: Condens. Matter

244

261

View full text Add to dashboard Cite

Abstract. Obtaining the eigenvalues and eigenvectors of large matrices is a key problem in electronic structure theory and many other areas of computational science. The computational effort formally scales as O(N 3 ) with the size of the investigated problem, N (e.g., the electron count in electronic structure theory), and thus often defines the system size limit that practical calculations cannot overcome. In many cases, more than just a small fraction of the possible eigenvalue/eigenvector pairs is needed, so that iterative solution strategies that focus only on few eigenvalues become ineffective. Likewise, it is not always desirable or practical to circumvent the eigenvalue solution entirely. We here review some current developments regarding dense eigenvalue solvers and then focus on the ELPA library, which facilitates the efficient algebraic solution of symmetric and Hermitian eigenvalue problems for dense matrices that have real-valued and complex-valued matrix entries, respectively, on parallel computer platforms. ELPA addresses standard as well as generalized eigenvalue problems, relying on the well documented matrix layout of the ScaLAPACK library but replacing all actual parallel solution steps with subroutines of its own. For these steps, ELPA significantly outperforms the corresponding ScaLAPACK routines and proprietary libraries that implement the ScaLAPACK interface (e.g., Intel's MKL). The most time-critical step is the reduction of the matrix to tridiagonal form and the corresponding backtransformation of the eigenvectors. ELPA offers both a one-step tridiagonalization (successive Householder transformations) and a two-step transformation that is more efficient especially towards larger matrices and larger numbers of CPU cores. ELPA is based on the MPI standard, with an early hybrid MPIOpenMPI implementation available as well. Scalability beyond 10,000 CPU cores for problem sizes arising in the electronic structure theory is demonstrated for current high-performance computer architectures such as Cray or Intel/Infiniband. For a ELPA -Eigenvalue Solutions for Electronic Structure Theory 2 matrix of dimension 260,000, scalability up to 295,000 CPU cores has been shown on BlueGene/P.

show abstract

“…[51,52,53,54] For the purposes of this paper, we note that ELPA relies upon an efficient implementation of the divide-and-conquer approach, the steps of which are reviewed in Ref. [22].…”

Section: The Eigenvalue Problemmentioning

confidence: 99%

The ELPA library: scalable parallel eigenvalue solutions for electronic structure theory and computational science

Marek

Blüm

Johanni

et al. 2014

J. Phys.: Condens. Matter

244

261

View full text Add to dashboard Cite

show abstract

“…We incorporated B as a new initial representations, and the necessary preprocessing into an existing implementation 1 , XMR [4], which already supported twisted block factorizations. We then obtained a parallel solver by adopting the approach layed out in [1].…”

Section: Mr 3 As An Svd Solvermentioning

confidence: 99%

Parallel Bidiagonal SVD via the Method of Multiple Relatively Robust Representations

Winkelmann

Bientinesi

2015

Proc Appl Math and Mech

View full text Add to dashboard Cite

A bidiagonal SVD solver based on the Method of Multiple Relatively Robust Representations (MR 3 ) promises to be an improvement over the current state of the art by providing lower asymptotic runtime and the ability to calculate a subset of singular triplets at reduced cost. Currently, no MR 3 based SVD solver is readily available. Using results by Willems and Lang [2], we provide an initial implementation of such a solver and compare it to LAPACK's Divide and Conquer solver. The Method of Multiple Relatively Robust RepresentationsRoughly speaking, MR 3 is similar to Inverse Iteration with the extra property that the computed eigenvectors do not require explicit orthogonalization. To achieve orthogonality of the eigenvectors, MR 3 treats clusters-eigenvalues with small relative distances-separately. If τ is just outside a cluster, then the shift T → T − Iτ has the effect of enlarging the relative distances of the clustered eigenvalues, so that the respective eigenvectors can be computed while maintaining orthogonality. To ensure accuracy, the shifted matrices are stored in robust representations, usually variants of LDL * decompositions. Unlike the Divide and Conquer method, MR 3 does not make any use of high level BLAS; therefore, in order to take advantage of multithreading, an explicit parallelization is required. In light of results by Petschow and Bientinesi [1], we use a task based parallelization approach which has the potential of yielding high parallel scalability. MR 3 as an SVD solverObtaining an accurate SVD solver from MR 3 is not a trivial matter. Willems and Lang [2] published a method based on the Golub-Kahan form T GK (B) of a bidiagonal matrix B; for details see Equation 8.6.3 in [3]. The Golub-Kahan form has the property that (σ, u, v) is a singular triplet of B if, and only if, (±σ, q ± ) is an eigenpair of T GK (B). Moreover, a singular vector of B can cheaply be obtained from the corresponding eigenvector q ± . Note, that two eigenvalues, −σ and +σ of T GK (B), correspond to one singular value σ of B. The entire SVD of B can be obtained from half the spectrum of the Golub-Kahan form, by calculating either all the negative or all the positive eigenvalues.The application of a black box MR 3 to the Golub-Kahan form does not yield an accurate solver. There are two problems which are related to loss of the Golub-Kahan structure. First, for robustness reasons, MR 3 starts out by shifting the input matrix to be definite. As a consequence, all eigenvalues of small magnitude become part of the same cluster. The problem is that the subspaces spanned by the eigenvectors corresponding to +σ and −σ can not be cleanly separated. As a result, the eigenvectors of small eigenvalues lose the Golub-Kahan structure and the extracted singular vectors are then not orthonormal.The other problem is that with repeated shifts, the LDL * representations may become increasingly dissimilar to the matrix that the decomposition is meant to represent. Such a phenomenon is caused by element growth: the decomposition uses ...

show abstract

“…Another paper compared the numerical performance of the netlib MRRR (as of LAPACK 3.3.2), netlib D&C, and an improved MRRR algorithm. The authors found that netlib MRRR produced unsatisfactory results for "not too few" of the test cases [Willems and Lang 2011]. The authors assert that the "best method for computing all eigenvalues" is a derivative of the QR algorithm, known as the dqds algorithm [Fernando and Parlett 1994;Parlett and Marques 1999], but suggest that it is avoided only because it is, in practice, found to be "very slow."…”

Section: Numerical Accuracymentioning

confidence: 99%

Restructuring the Tridiagonal and Bidiagonal QR Algorithms for Performance

Zee

Geijn

Quintana-Ortí

2014

ACM Trans. Math. Softw.

View full text Add to dashboard Cite

We show how both the tridiagonal and bidiagonal QR algorithms can be restructured so that they become rich in operations that can achieve near-peak performance on a modern processor. The key is a novel, cache-friendly algorithm for applying multiple sets of Givens rotations to the eigenvector/singular vector matrix. This algorithm is then implemented with optimizations that (1) leverage vector instruction units to increase floating-point throughput, and (2) fuse multiple rotations to decrease the total number of memory operations. We demonstrate the merits of these new QR algorithms for computing the Hermitian eigenvalue decomposition (EVD) and singular value decomposition (SVD) of dense matrices when all eigenvectors/singular vectors are computed. The approach yields vastly improved performance relative to the traditional QR algorithms for these problems and is competitive with two commonly used alternativesCuppen's Divide and Conquer algorithm and the Method of Multiple Relatively Robust Representationswhile inheriting the more modest O(n) workspace requirements of the original QR algorithms. Since the computations performed by the restructured algorithms remain essentially identical to those performed by the original methods, robust numerical properties are preserved.

show abstract

A Framework for the $MR^3$ Algorithm: Theory and Implementation

Cited by 10 publications

References 31 publications

The ELPA library: scalable parallel eigenvalue solutions for electronic structure theory and computational science

The ELPA library: scalable parallel eigenvalue solutions for electronic structure theory and computational science

Parallel Bidiagonal SVD via the Method of Multiple Relatively Robust Representations

Restructuring the Tridiagonal and Bidiagonal QR Algorithms for Performance

Contact Info

Product

Resources

About