Matrices with low-rank structure are ubiquitous in scientific computing. Choosing an appropriate rank is a key step in many computational algorithms that exploit low-rank structure. However, estimating the rank has been done largely in an ad-hoc fashion in previous studies. In this work we develop a randomized algorithm for estimating the numerical rank of a matrix. The algorithm is based on sketching the matrix with random matrices from both left and right; the key fact is that with high probability, the sketches preserve the orders of magnitude of the leading singular values. The rank can hence be taken to be the number of singular values of the sketch that are larger than the prescribed threshold. For an m × n (m ≥ n) matrix of numerical rank r, the algorithm runs with complexity O(mn log n + r 3 ), or less for structured matrices. The steps in the algorithm are required as a part of many low-rank algorithms, so the additional work required to estimate the rank can be even smaller in practice. Numerical experiments illustrate the speed and robustness of our rank estimator.
We describe two algorithms to efficiently solve regularized linear least squares systems based on sketching. The algorithms compute preconditioners for min Ax − b 2 2 + λ x 2 2 , where A ∈ R m×n and λ > 0 is a regularization parameter, such that LSQR converges in O(log(1/ )) iterations for accuracy. We focus on the context where the optimal regularization parameter is unknown, and the system must be solved for a number of parameters λ. Our algorithms are applicable in both the underdetermined m n and the overdetermined m n setting. Firstly, we propose a Cholesky-based sketch-to-precondition algorithm that uses a 'partly exact' sketch, and only requires one sketch for a set of N regularization parameters λ. The complexity of solving for N parameters is O(mn log(max(m, n)) + N (min(m, n) 3 + mn log(1/ ))). Secondly, we introduce an algorithm that uses a sketch of size O(sd λ (A)) for the case where the statistical dimension sd λ (A) min(m, n). The scheme we propose does not require the computation of the Gram matrix, resulting in a more stable scheme than existing algorithms in this context. We can solve for N values of λ i in O(mn log(max(m, n)) + min(m, n) sd min λ i (A) 2 + N mn log(1/ )) operations.
Sketch-and-precondition techniques are popular for solving large least squares (LS) problems of the form Ax = b with A ∈ R m×n and m ≫ n. This is where A is "sketched" to a smaller matrix SA with S ∈ R ⌈cn⌉×m for some constant c > 1 before an iterative LS solver computes the solution to Ax = b with a right preconditioner P , where P is constructed from SA. Popular sketch-and-precondition LS solvers are Blendenpik and LSRN. We show that the sketchand-precondition technique is not numerically stable for ill-conditioned LS problems. Instead, we propose using an unpreconditioned iterative LS solver on (AP )y = b with x = P y when accuracy is a concern. Provided the condition number of A is smaller than the reciprocal of the unit round-off, we show that this modification ensures that the computed solution has a comparable backward error to the iterative LS solver applied to a well-conditioned matrix. Using smoothed analysis, we model floating-point rounding errors to provide a convincing argument that our modification is expected to compute a backward stable solution even for arbitrarily ill-conditioned LS problems.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.