How to Find a Good Submatrix

Goreinov, Sergei A.; Oseledets, Ivan; Savostyanov, Dmitry; Тыртышников, Е. Е.; Замарашкин, Н. Л.

doi:10.1142/9789812836021_0015

Cited by 156 publications

(207 citation statements)

References 4 publications

Supporting

Mentioning

206

Contrasting

Order By: Relevance

“…One can alternatively approximate leading singular spaces by applying the algorithm of [GOSTZ10], devised for the approximation of the so called CUR decomposition of a matrix. The algorithm is heuristic, but consistently converges fast according to its extensive tests by the authors.…”

Section: Leading Singular Spaces Via the Maximum Volumementioning

confidence: 99%

“…In the authors' tests, the iterative algorithm of [GOSTZ10] has consistently produced ρ × ρ submatrices of the matrix A that have reasonably bounded ratios ν. This work is linked to our study because a nearly optimal rank-ρ approximation CA −1 11 R to the matrix A induces close approximations by the matrices C and CA −1 11 to n × ρ matrix bases of the leading singular space T ρ,A T .…”

Section: −1mentioning

confidence: 99%

See 1 more Smart Citation

New studies of randomized augmentation and additive preprocessing

Pan

Zhao

2017

Linear Algebra and its Applications

View full text Add to dashboard Cite

Abstract• A standard Gaussian random matrix has full rank with probability 1 and is wellconditioned with a probability quite close to 1 and converging to 1 fast as the matrix deviates from square shape and becomes more rectangular.• If we append sufficiently many standard Gaussian random rows or columns to any matrix A, such that ||A|| = 1, then the augmented matrix has full rank with probability 1 and is well-conditioned with a probability close to 1, even if the matrix A is rank deficient or ill-conditioned.• We specify and prove these properties of augmentation and extend them to additive preprocessing, that is, to adding a product of two rectangular Gaussian matrices.• By applying our randomization techniques to a matrix that has numerical rank ρ, we accelerate the known algorithms for the approximation of its leading and trailing singular spaces associated with its ρ largest and with all its remaining singular values, respectively.• Our algorithms use much fewer random parameters and run much faster when various random sparse and structured preprocessors replace Gaussian. Empirically the outputs of the resulting algorithms is as accurate as the outputs under Gaussian preprocessing.• Our novel duality techniques provides formal support, so far missing, for these empirical observations and opens door to derandomization of our preprocessing and to further acceleration and simplification of our algorithms by using more efficient sparse and structured preprocessors.• Our techniques and our progress can be applied to various other fundamental matrix computations such as the celebrated low-rank approximation of a matrix by means of random sampling. 1Key Words: Randomized matrix algorithms; Gaussian random matrices; Singular spaces of a matrix; Duality; Derandomization; Sparse and structured preprocessors 1 Introduction Randomized augmentation: outlineA standard Gaussian m × n random matrix, G (hereafter referred to just as Gaussian), has full rank with probability 1 (see Theorem B.1). Furthermore the expected spectral norms ||G|| and ||G + ||, G + denoting the Moore-Penrose generalized inverse, satisfy the following estimates (see Theorems B.2 and B.3):• E(||G||) ≈ 2 √ h, for h = max{m, n}, and|m−n| provided that l = min{m, n}, m = n, and e = 2.71828 . . . . Thus, for moderate or reasonably large integers m and n, the matrix G can be considered wellconditioned with the confidence growing fast as the integer |m − n| increases from 0. By virtue of part 2 of Theorem B.3, the matrix G can be viewed as well-conditioned even for m = n, although with a grain of salt, depending on context. Motivated by this information, we append sufficiently but reasonably many Gaussian rows or columns to any matrix A, possibly rank deficient or ill-conditioned, but normalized, such that ||A|| = 1. (Our approach requires attention to various pitfalls, and in particular it fails without normalization of an input matrix.) Then we prove that the cited properties of a Gaussian matrix also hold for the augmented matrix K and similarly for th...

show abstract

Section: Leading Singular Spaces Via the Maximum Volumementioning

confidence: 99%

Section: −1mentioning

confidence: 99%

New studies of randomized augmentation and additive preprocessing

Pan

Zhao

2017

Linear Algebra and its Applications

View full text Add to dashboard Cite

show abstract

“…It was shown that an appropriate way of finding a good subset for the CSSP consists in selecting a number of columns such that their volume is maximal. This criterion is also referred to as maximum volume (Max-Vol) criterion [6,13,14]. Consequently, a good subset is one that maximizes the volume of the parallelepiped, i.e.…”

Section: Deterministic Selection (Dcur)mentioning

confidence: 99%

Deterministic CUR for Improved Large-Scale Data Analysis: An Empirical Study

Thurau¹,

Kersting

Bauckhage³

2012

Proceedings of the 2012 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Low-rank approximations which are computed from selected rows and columns of a given data matrix have attracted considerable attention lately. They have been proposed as an alternative to the SVD because they naturally lead to interpretable decompositions which was shown to be successful in application such as fraud detection, fMRI segmentation, and collaborative filtering. The CUR decomposition of large matrices, for example, samples rows and columns according to a probability distribution that depends on the Euclidean norm of rows or columns or on other measures of statistical leverage. At the same time, there are various deterministic approaches that do not resort to sampling and were found to often yield factorization of superior quality with respect to reconstruction accuracy. However, these are hardly applicable to large matrices as they typically suffer from high computational costs. Consequently, many practitioners in the field of data mining have abandon deterministic approaches in favor of randomized ones when dealing with today's large-scale data sets. In this paper, we empirically disprove this prejudice. We do so by introducing a novel, linear-time, deterministic CUR approach that adopts the recently introduced Simplex Volume Maximization approach for column selection. The latter has already been proven to be successful for NMF-like decompositions of matrices of billions of entries. Our exhaustive empirical study on more than 30 synthetic and real-world data sets demonstrates that it is also beneficial for CUR-like decompositions. Compared to other deterministic CUR-like methods, it provides comparable reconstruction quality but operates much faster so that it easily scales to matrices of billions of elements. Compared to sampling-based methods, it provides competitive reconstruction quality while staying in the same run-time complexity class.

show abstract

“…If we define the volume of a square matrix as the modulus of its determinant, then our target is to find a B of maximum volume. For such a matrix, Goreinov et al [21] have shown that all the entries in |N B −1 | will be less than or equal to 1.0. They have proposed an algorithm for finding such a matrix, but it is impractical for large matrices because of the combinatorial complexity.…”

Section: The Choice Of the Basis Bmentioning

confidence: 99%

Preconditioning Linear Least-Squares Problems by Identifying a Basis Matrix

Arioli¹,

Duff²

2015

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

Abstract. We study the solution of the linear least-squares problem minx b − Ax 2 2 where the matrix A ∈ R m×n (m ≥ n) has rank n and is large and sparse. We assume that A is available as a matrix, not an operator. The preconditioning of this problem is difficult because the matrix A does not have the properties of differential problems that make standard preconditioners effective. Incomplete Cholesky techniques applied to the normal equations do not produce a well-conditioned problem. We attempt to bypass the ill-conditioning by finding an n × n nonsingular submatrix B of A that reduces the Euclidean norm of AB −1 . We use B to precondition a symmetric quasi-definite linear system whose condition number is then independent of the condition number of A and has the same solution as the original least-squares problem. We illustrate the performance of our approach on some standard test problems and show it is competitive with other approaches.

show abstract

How to Find a Good Submatrix

Cited by 156 publications

References 4 publications

New studies of randomized augmentation and additive preprocessing

New studies of randomized augmentation and additive preprocessing

Deterministic CUR for Improved Large-Scale Data Analysis: An Empirical Study

Preconditioning Linear Least-Squares Problems by Identifying a Basis Matrix

Contact Info

Product

Resources

About