Sampling and multilevel coarsening algorithms for fast matrix approximations

Abstract

Summary This paper addresses matrix approximation problems for matrices that are large, sparse, and/or representations of large graphs. To tackle these problems, we consider algorithms that are based primarily on coarsening techniques, possibly combined with random sampling. A multilevel coarsening technique is proposed, which utilizes a hypergraph associated with the data matrix and a graph coarsening strategy based on column matching. We consider a number of standard applications of this technique as well as a few new ones. Among standard applications, we first consider the problem of computing partial singular value decomposition, for which a combination of sampling and coarsening yields significantly improved singular value decomposition results relative to sampling alone. We also consider the column subset selection problem, a popular low‐rank approximation method used in data‐related applications, and show how multilevel coarsening can be adapted for this problem. Similarly, we consider the problem of graph sparsification and show how coarsening techniques can be employed to solve it. We also establish theoretical results that characterize the approximation error obtained and the quality of the dimension reduction achieved by a coarsening step, when a proper column matching strategy is employed. Numerical experiments illustrate the performances of the methods in a few applications.

Sparse Graph Based Sketching for Fast Numerical Linear Algebra

“…Many modern applications involving large-dimensional data resort to data reduction techniques such as matrix approximation [14,33] or compression [29] in order to lower the amount of data processed, and to speed up computations. In recent years, randomized sketching techniques have attracted much attention in the literature for computing such matrix approximations, due to their high efficiency and well-established theoretical guarantees [19,14,34].…”

“…Sketching methods have been used to speed up numerical linear algebra problems such as least squares regression, low-rank approximation, matrix multiplication, and approximating leverage scores [28,11,14,34,32]. These primitives as well as improved matrix algorithms have been developed based on sketching for various tasks in machine learning [5,37], signal processing [9,26], scientific computing [33], and optimization [25,23].…”

Projection-based QLP Algorithm for Efficiently Computing Low-Rank Approximation of Matrices

“…M ATRICES with low-rank structure are omnipresent in signal processing, data analysis, scientific computing and machine learning applications including system identification [1], subspace clustering [2], matrix completion [3], background subtraction [4], [5], least-squares regression [6], hyperspectral imaging [7], [8], anomaly detection [9], [10], subspace estimation over networks [11], genomics [12], [13], tensor decompositions [14], and sparse matrix problems [15].…”

Sparse graph based sketching for fast numerical linear algebra

“…to speed up numerical linear algebra problems such as least squares regression, low-rank approximation, matrix multiplication, and approximating leverage scores [28,11,14,34,32]. These primitives as well as improved matrix algorithms have been developed based on sketching for various tasks in machine learning [5,37], signal processing [9,26], scientific computing [33], and optimization [25,23].…”

“…But, for a good rank-k approximation, the sketch size will have to be m = O k 2 2 [34]. Applications involving low-rank approximation of large sparse matrices include Latent Semantic Indexing (LSI) [20], matrix completion problems [15], subspace tracking in signal processing [36] and others [33].…”