Randomized QR with Column Pivoting

Abstract

Abstract. The dominant contribution to communication complexity in factorizing a matrix using QR with column pivoting is due to column-norm updates that are required to process pivot decisions. We use randomized sampling to approximate this process which dramatically reduces communication in column selection. We also introduce a sample update formula to reduce the cost of sampling trailing matrices. Using our column selection mechanism we observe results that are comparable in quality to those obtained from the QRCP algorithm, but with performance near unpivoted QR. We also demonstrate strong parallel scalability on shared memory multiple core systems using an implementation in Fortran with OpenMP.This work immediately extends to produce low-rank truncated approximations of large matrices. We propose a truncated QR factorization with column pivoting that avoids trailing matrix updates which are used in current implementations of level-3 BLAS QR and QRCP. Provided the truncation rank is small, avoiding trailing matrix updates reduces approximation time by nearly half. By using these techniques and employing a variation on Stewart's QLP algorithm, we develop an approximate truncated SVD that runs nearly as fast as truncated QR.Keywords. QR factorization, column pivoting, random sampling, sample update, blocked algorithm, low-rank approximation, truncated SVD 1. Introduction. We explore a variation of QR with Column Pivoting (QRCP) using randomized sampling (RQRCP) to process blocks of pivots. Magnitudes of trailing column norms are detected using Gaussian random compression matrices to produce smaller sample matrices. We analyze the probability distributions of sample column norms to justify the internal updating computation used to select blocks of column pivots. P.G. Martinsson [16] independently developed a very similar approach in parallel to this research. The primary difference is our introduction of a sample update formula which reduces matrix-multiplication complexity required to process the full matrix factorization by one third of what would be required by re-sampling after processing each block of column pivots.We also extend this method of factorization to produce truncated low-rank approximations. We propose an implementation that avoids the trailing update computation on the full matrix. This further reduces time spent in matrix multiplication to nearly half of what would be required by a truncated version with trailing update that also employs one of our sample update formulas. Furthermore, the Truncated Randomized QR with Column Pivoting algorithm (TRQRCP) immediately extends to approximate the truncated SVD using a variation on Stewart's QLP algorithm [20].We are able to achieve matrix factorizations of similar quality to standard QRCP while retaining communication complexity of unpivoted QR. Algorithms have been implemented and tested in Fortran with OpenMP on shared-memory 24-core systems. Our performance experiments compare these algorithms against LAPACK subroutines linked with the Intel Math...

Efficient Randomized Algorithms for the Fixed-Precision Low-Rank Matrix Approximation

“…Remark 3.2. Algorithm 1 can also be combined with the trQRCP algorithm [14]. Although trQRCP generates a column permutation matrix as well, it does not affect the Frobenius norm of each partial or the entire matrix of B.…”

Flip-flop spectrum-revealing QR factorization and its applications to singular value decomposition

“…with an approximation quality closely related to the error term in R 22 . The Randomized QRCP (RQRCP) algorithm [17,75] is a more efficient variant of Algorithm 4. RQRCP generates a Gaussian random matrix Ω ∈ N (0, 1) Form the initial sample matrix B = ΩA and initialize Π = I n .…”

“…The pivoted QLP decomposition is extensively analyzed in Huckaby and Chan [34,35]. More recently, Duersch and Gu developed a much more efficient variant of the pivoted QLP decomposition, TUXV, and demonstrated its remarkable quality as a low-rank approximation empirically without a rigorous theoretical justification of TUXV's success [17].…”

“…In this paper, we present Flip-Flop SRQR, a slightly different variant of TUXV of Duersch and Gu [17]. Like TUXV, Flip-Flop SRQR requires the most effort for computing a partial QR factorization using a truncated randomized QRCP (TRQRCP) and a partial LQ factorization.…”

Compressed Randomized UTV Decompositions for Low-Rank Matrix Approximations

“…We assess the quality of lowrank approximation by reconstructing a gray-scale image of a differential gear of size 1280 × 804, taken from [55], using CoR-UTV, truncated QRCP, and the truncated SVD by using (widely recommended) PROPACK package [110]. The PROPACK function provides an efficient algorithm to compute a specified number of largest singular values and corresponding singular vectors of a given matrix by making use of the Lanczos bidiagonalization algorithm with partial reorthogonalization, which is suitable for approximating large low-rank matrices.…”