Novel Newton algorithms for the Hermitian eigenvalue problem

Abstract

We present three related algorithms for iteratively computing all the eigenvectors of a Hermitian matrix. The algorithms are based on the idea of applying Newton updates (performed on a complex projective space) to individual eigenvectors at each iteration. The advantage of these Newton updates is that they have a cubic rate of convergence. The difference between the algorithms is how they prevent the individual updates from converging to the same eigenvector. The first algorithm fmds the eigenvectors sequentially, and uses a novel form of deflation in order to ensure all the eigenvectors are found. Rather than modi@ the matrix directly, which introduces large errors if the matrix is ill-conditioned, deflation is achieved by restricting the Newton updates to lie in a subspace orthogonal to all previously found eigenvectors. The other algorithms estimate all the eigenvectors at once. At each iteration, they sweep through all the estimates, performing a Newton update on each estimate once per sweep. Orthogonality is maintained by explicit re-orthogonalisation after each update, which also serves to improve the asymptotic rate of convergence of the algorithms.

Generalizations of the Rayleigh Quotient Iteration for the iterative refinement of the eigenvectors of real symmetric matrices

“…In the algorithm of [4,5] this was achieved by a full orthogonalisation of the set of vectors after performing RQI on each in parallel. In an algorithm in [1,6] it was done by orthogonalisation of the full set after each RQI update of one vector. Another closely related algorithm in [1,6] also achieved this by replacing the least recently updated vector by a vector orthogonal to all the others before performing the RQI update on it.…”

“…In an algorithm in [1,6] it was done by orthogonalisation of the full set after each RQI update of one vector. Another closely related algorithm in [1,6] also achieved this by replacing the least recently updated vector by a vector orthogonal to all the others before performing the RQI update on it. In [2] it was discussed how these algorithms relate to the well known shifted QR algorithm.…”

“…In this work we show that local cubic convergence of all eigenvectors can be achieved with more simple operations than full orthogonalisation. This comes at the price of losing possible good global convergence properties in the algorithms of [1,2,6], but we maintain fast local convergence which makes the algorithms useful for adaptive applications.…”

“…Recently and independently the authors looked at developing algorithms in which all eigenvalue-eigenvector pairs simultaneously converge cubically [1,2,4,5,6]. These algorithms took the RQI as a starting point and then developed generalisations to the full matrix case.…”

Analysis of Dynamical Systems for Generalized Principal and Minor Component Extraction

“…Over the last few years, several neural networks and learning rules for performing generalized eigendecomposition [10,11,12] have been proposed. In this paper we propose and analyze new dynamical systems that can be applied to nonsymmetric matrices or symmetric matrices which are not positive definite.…”