New studies of randomized augmentation and additive preprocessing

Abstract

Abstract• A standard Gaussian random matrix has full rank with probability 1 and is wellconditioned with a probability quite close to 1 and converging to 1 fast as the matrix deviates from square shape and becomes more rectangular.• If we append sufficiently many standard Gaussian random rows or columns to any matrix A, such that ||A|| = 1, then the augmented matrix has full rank with probability 1 and is well-conditioned with a probability close to 1, even if the matrix A is rank deficient or ill-conditioned.• We specify and prove these properties of augmentation and extend them to additive preprocessing, that is, to adding a product of two rectangular Gaussian matrices.• By applying our randomization techniques to a matrix that has numerical rank ρ, we accelerate the known algorithms for the approximation of its leading and trailing singular spaces associated with its ρ largest and with all its remaining singular values, respectively.• Our algorithms use much fewer random parameters and run much faster when various random sparse and structured preprocessors replace Gaussian. Empirically the outputs of the resulting algorithms is as accurate as the outputs under Gaussian preprocessing.• Our novel duality techniques provides formal support, so far missing, for these empirical observations and opens door to derandomization of our preprocessing and to further acceleration and simplification of our algorithms by using more efficient sparse and structured preprocessors.• Our techniques and our progress can be applied to various other fundamental matrix computations such as the celebrated low-rank approximation of a matrix by means of random sampling. 1Key Words: Randomized matrix algorithms; Gaussian random matrices; Singular spaces of a matrix; Duality; Derandomization; Sparse and structured preprocessors 1 Introduction Randomized augmentation: outlineA standard Gaussian m × n random matrix, G (hereafter referred to just as Gaussian), has full rank with probability 1 (see Theorem B.1). Furthermore the expected spectral norms ||G|| and ||G + ||, G + denoting the Moore-Penrose generalized inverse, satisfy the following estimates (see Theorems B.2 and B.3):• E(||G||) ≈ 2 √ h, for h = max{m, n}, and|m−n| provided that l = min{m, n}, m = n, and e = 2.71828 . . . . Thus, for moderate or reasonably large integers m and n, the matrix G can be considered wellconditioned with the confidence growing fast as the integer |m − n| increases from 0. By virtue of part 2 of Theorem B.3, the matrix G can be viewed as well-conditioned even for m = n, although with a grain of salt, depending on context. Motivated by this information, we append sufficiently but reasonably many Gaussian rows or columns to any matrix A, possibly rank deficient or ill-conditioned, but normalized, such that ||A|| = 1. (Our approach requires attention to various pitfalls, and in particular it fails without normalization of an input matrix.) Then we prove that the cited properties of a Gaussian matrix also hold for the augmented matrix K and similarly for th...

CUR Low Rank Approximation of a Matrix at Sublinear Cost

“…This continues our earlier study of dual problems of matrix computations with random input, in particular Gaussian elimination where randomization replaces pivoting (see [PQY15], [PZ17a], and [PZ17b]). We further advance this approach in [PLSZa], [PLa], [PLb], and [LPSa].…”

Random multipliers numerically stabilize Gaussian and block Gaussian elimination: Proofs and an extension to low-rank approximation

“…A natural research challenge is the combination of our randomized multiplicative preprocessing with randomized augmentation and additive preprocessing, studied in [28,29,31,32,38,35,37]. under some mild assumptions on the positive oversampling integer p. The above bounds show that low-rank approximations of high quality can be obtained by using a reasonably small oversampling integer parameter p, say p = 20, but they do not apply where p ≤ 1.…”

Low Rank Approximation at Sublinear Cost

“…[PLSZ16] and [PLSZ17] were the first papers that provided formal support for dual accurate randomized LRA computations performed at sub-linear cost (in these papers such computations are called superfast). The earlier papers [PQY15], [PLSZ16], [PZ17a], and [PZ17b] studied duality for other fundamental matrix computations besides LRA, while the paper [PLb] has extended our study to a sub-linear cost dual algorithm for the popular problem of Linear Least Squares Regression and confirmed accuracy of this solution by the results of numerical experiments.…”

“…Impact of our study, its extensions and by-products: (i) Our duality approach enables new insight into some fundamental matrix computations besides LRA: [PQY15], [PZ17a], and [PZ17b] provide formal support for empirical efficiency of dual Gaussian elimination with no pivoting, while [PLb] proposes a sub-linear cost modification of Sarlós' algorithm of 2006 and then proves that whp it outputs nearly optimal solution of the highly important problem of Linear Least Squares Regression (LLSR) provided that its input is random. Then again this formal proof is in good accordance with the test results.…”