Computing the Square Root of a Low-Rank Perturbation of the Scaled Identity Matrix

Abstract

We consider the problem of computing the square root of a perturbation of the scaled identity matrix, A = αIn + U V * , where U and V are n × k matrices with k ≤ n. This problem arises in various applications, including computer vision and optimization methods for machine learning. We derive a new formula for the pth root of A that involves a weighted sum of powers of the pth root of the k × k matrix αI k + V * U . This formula is particularly attractive for the square root, since the sum has just one term when p = 2. We also derive a new class of Newton iterations for computing the square root that exploit the low-rank structure. We test these new methods on random matrices and on positive definite matrices arising in applications. Numerical experiments show that the new approaches can yield much smaller residual than existing alternatives and can be significantly faster when the perturbation U V * has low rank.

Low‐rank updates of matrix square roots

“…We stress that our method has an advantage over Fasi et al, 9 when applied to SHAMPOO. Fasi et al, 9 can only handle perturbations of the identity, so when applied to compute the square root of $$$ {\mathbf{L}}_{t+1} $$$ it cannot use the square root of $$$ {\mathbf{L}}_t $$$ (which is available from the previous iteration).…”

“…We stress that our method has an advantage over Fasi et al, 9 when applied to SHAMPOO. Fasi et al, 9 can only handle perturbations of the identity, so when applied to compute the square root of $$$ {\mathbf{L}}_{t+1} $$$ it cannot use the square root of $$$ {\mathbf{L}}_t $$$ (which is available from the previous iteration). Our algorithm, on the other hand, can use the fact that $$$ {\mathbf{L}}_{t+1}={\mathbf{L}}_t+{\mathbf{G}}_t{\mathbf{G}}_t^T $$$ for a low rank update of the previous iteration.…”

“…Our algorithm, on the other hand, can use the fact that $$$ {\mathbf{L}}_{t+1}={\mathbf{L}}_t+{\mathbf{G}}_t{\mathbf{G}}_t^T $$$ for a low rank update of the previous iteration. If the matrices are explicitly held, Fasi et al, 9 will have a cost per iteration that grows linearly with iteration count, while with our algorithm the cost will stay constant.…”

“…The work most similar to ours is Reference 9. In that article, the authors consider the problem of computing the square root of a matrix of the form $$$ \alpha \mathbf{I}+\mathbf{U}{\mathbf{V}}^T $$$, as a correction of $$$ \sqrt{\alpha}\mathbf{I} $$$.…”

“…The authors circumvent this issue by suggesting another formula for the square root, or by using a Newton iteration. In a way, the algorithm in Reference 9 is more general than our method since it allows non‐symmetric updates. However, in another way it is less general: the matrix to be perturbed must be a scaled identity matrix.…”