Faster Subset Selection for Matrices and Applications

Avron, Haim; Boutsidis, Christos

doi:10.1137/120867287

Cited by 77 publications

(129 citation statements)

References 51 publications

Supporting

Mentioning

126

Contrasting

Order By: Relevance

“…We restrict our attention to approximation algorithms for these problems and refer the reader to Pukelsheim (2006) for a broad survey on experimental design. Avron and Boutsidis (2013) studied the Aand E-optimal design problems and analyzed various combinatorial algorithms and algorithms based on volume sampling, and achieved approximation ratio n−d+1 k−d+1 . Wang et al (2016) found connections between optimal design and matrix sparsification, and used these connections to obtain a (1 + )-approximation when k ≥ d 2 , and also approximation algorithms under certain technical assumptions.…”

Section: Related Workmentioning

confidence: 99%

“…[1] The ratios are tight with matching integrality gap of the convex relaxation (1)-(3). Spielman and Srivastava (2011)) and column subset selection (Avron and Boutsidis (2013)) in numerical linear algebra. In this work, we consider the optimization problem of choosing the representative subset that aims to optimize the A-optimality criterion in experimental design.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Proportional Volume Sampling and Approximation Algorithms for A-Optimal Design

Nikolov¹,

Singh²,

Tantipongpipat³

2019

Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms

View full text Add to dashboard Cite

We study the A-optimal design problem where we are given vectors v 1 , . . . , v n ∈ R d , an integer k ≥ d, and the goal is to select a set S of k vectors that minimizes the trace ofthe problem is an instance of optimal design of experiments in statistics (Pukelsheim (2006)) where each vector corresponds to a linear measurement of an unknown vector and the goal is to pick k of them that minimize the average variance of the error in the maximum likelihood estimate of the vector being measured. The problem also finds applications in sensor placement in wireless networks (Joshi and Boyd (2009)), sparse least squares regression (Boutsidis et al. (2011)), feature selection for k-means clustering (Boutsidis and Magdon-Ismail (2013)), and matrix approximation Mattheij (2007, 2011); Avron and Boutsidis (2013)). In this paper, we introduce proportional volume sampling to obtain improved approximation algorithms for A-optimal design. Given a matrix, proportional volume sampling involves picking a set of columns S of size k with probability proportional to µ(S) times det( i∈S v i v i ) for some measure µ. Our main result is to show the approximability of the A-optimal design problem can be reduced to approximate independence properties of the measure µ. We appeal to hard-core distributions as candidate distributions µ that allow us to obtain improved approximation algorithms for the A-optimal design. Our results include a d-approximation when k = d, an (1 + )-approximation when k = Ω d + 1 2 log 1 and k k−d+1approximation when repetitions of vectors are allowed in the solution. We also consider generalization of the problem for k ≤ d and obtain a k-approximation.We also show that the proportional volume sampling algorithm gives approximation algorithms for other optimal design objectives (such as D-optimal design Singh and Xie (2018) and generalized ratio objective Mariet and Sra (2017)) matching or improving previous best known results. Interestingly, we show that a similar guarantee cannot be obtained for the E-optimal design problem. We also show that the A-optimal design problem is NP-hard to approximate within a fixed constant when k = d.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Proportional Volume Sampling and Approximation Algorithms for A-Optimal Design

Nikolov¹,

Singh²,

Tantipongpipat³

2019

Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms

View full text Add to dashboard Cite

show abstract

“…Finally, recent work [2] describes a deterministic T V k + O (nk (n − r)) time algorithm that guarantees an approximation error…”

Section: Running Timesmentioning

confidence: 99%

Near-Optimal Column-Based Matrix Reconstruction

Boutsidis¹,

Drineas²,

2014

Self Cite

View full text Add to dashboard Cite

Abstract. We consider low-rank reconstruction of a matrix using a subset of its columns and we present asymptotically optimal algorithms for both spectral norm and Frobenius norm reconstruction. The main tools we introduce to obtain our results are: (i) the use of fast approximate SVD-like decompositions for column-based matrix reconstruction, and (ii) two deterministic algorithms for selecting rows from matrices with orthonormal columns, building upon the sparse representation theorem for decompositions of the identity that appeared in [1].

show abstract

“…7,18,20,29 Greedy subset selection algorithms have also been proposed based on the right singular vectors of the matrix. 30,31 Note that these methods may be expensive when rank k is not small since they require the top-k singular vectors. An alternative solution to the CSSP is to use the coarsened graph consisting of the columns obtained by graph coarsening.…”

Section: Column Subset Selectionmentioning

confidence: 99%

Sampling and multilevel coarsening algorithms for fast matrix approximations

Ubaru

Saad

2019

Numerical Linear Algebra App

View full text Add to dashboard Cite

Summary This paper addresses matrix approximation problems for matrices that are large, sparse, and/or representations of large graphs. To tackle these problems, we consider algorithms that are based primarily on coarsening techniques, possibly combined with random sampling. A multilevel coarsening technique is proposed, which utilizes a hypergraph associated with the data matrix and a graph coarsening strategy based on column matching. We consider a number of standard applications of this technique as well as a few new ones. Among standard applications, we first consider the problem of computing partial singular value decomposition, for which a combination of sampling and coarsening yields significantly improved singular value decomposition results relative to sampling alone. We also consider the column subset selection problem, a popular low‐rank approximation method used in data‐related applications, and show how multilevel coarsening can be adapted for this problem. Similarly, we consider the problem of graph sparsification and show how coarsening techniques can be employed to solve it. We also establish theoretical results that characterize the approximation error obtained and the quality of the dimension reduction achieved by a coarsening step, when a proper column matching strategy is employed. Numerical experiments illustrate the performances of the methods in a few applications.

show abstract

Faster Subset Selection for Matrices and Applications

Cited by 77 publications

References 51 publications

Proportional Volume Sampling and Approximation Algorithms for A-Optimal Design

Proportional Volume Sampling and Approximation Algorithms for A-Optimal Design

Near-Optimal Column-Based Matrix Reconstruction

Sampling and multilevel coarsening algorithms for fast matrix approximations

Contact Info

Product

Resources

About