Efficient Volume Sampling for Row/Column Subset Selection

Abstract

We give efficient algorithms for volume sampling, i.e., for picking k-subsets of the rows of any given matrix with probabilities proportional to the squared volumes of the simplices defined by them and the origin (or the squared volumes of the parallelepipeds defined by these subsets of rows). This solves an open problem from the monograph on spectral algorithms by Kannan and Vempala (see Section 7.4 of [15], also implicit in [1,5]).Our first algorithm for volume sampling k-subsets of rows from an m-by-n matrix runs in O(kmn ω log n) arithmetic operations and a second variant of it for (1 + ǫ)-approximate volume sampling runs in O(mn log m · k 2 /ǫ 2 + m log ω m · k 2ω+1 /ǫ 2ω · log(kǫ −1 log m)) arithmetic operations, which is almost linear in the size of the input (i.e., the number of entries) for small k.Our efficient volume sampling algorithms imply the following results for low-rank matrix approximation:1. Given A ∈ R m×n , in O(kmn ω log n) arithmetic operations we can find k of its rows such that projecting onto their span gives a √ k + 1-approximation to the matrix of rank k closest to A under the Frobenius norm. This improves the O(k √ log k)-approximation of Boutsidis, Drineas and Mahoney [1] and matches the lower bound shown in [5]. The method of conditional expectations gives a deterministic algorithm with the same complexity. The running time can be improved to O(mn log m·k 2 /ǫ 2 +m log ω m·k 2ω+1 /ǫ 2ω ·log(kǫ −1 log m)) at the cost of losing an extra (1 + ǫ) in the approximation factor.2. The same rows and projection as in the previous point give a (k + 1)(n − k)-approximation to the matrix of rank k closest to A under the spectral norm. In this paper, we show an almost matching lower bound of √ n, even for k = 1.

Near-Optimal Column-Based Matrix Reconstruction

“…We show that O(k/ǫ) columns contain a rank-k subspace which reconstructs A to relative error, and we present the first sub-SVD (in terms of running time) randomized algorithm to identify these columns. This matches the Ω(k/ǫ) lower bound in [8] and improves the best known upper bound of O(k log k + k/ǫ) [6,8,12,23]. …”

“…There is considerable interest (e.g. [4,6,8,9,11,15,19,20,21]) in determining a minimum set of r ≪ n columns of A which is approximately as good as A k at reconstructing A. Such columns are important for interpreting data [21], building robust machine learning algorithms [4], feature selection, etc.…”

“…We present polynomial-time (deterministic and randomized) algorithms with approximation error asymptotically matching a lower bound proven in this work. Prior work had focused on the r = k case and presented near-optimal polynomial-time algorithms [6,17].…”

Movie shot selection preserving narrative properties

“…Thus, approximate algorithms with lower computational complexity have been presented in the relevant literature, with the goal of finding a suboptimal but acceptable solution. The proposed approaches include randomized, deterministic or hybrid methods, using SVD sparse approximation [16], random selection of matrix columns, based on a probability distribution, which is later refined deterministically [10], greedy recursive computation of the reconstruction error, initialized with random projections of the matrix columns [17], column subset selection with probabilities proportional to the squared volumes of the parallelepipeds defined by these subsets [18], etc.…”

Real stable polynomials and matroids: optimization and counting

“…These include the problem of sampling from a DPP, computing its partition function and the MAP-inference problem which asks to find the set of highest probability (or equivalently to find the largest coefficient of q(x)). For the case of unconstrained DPPs problems (1) and (2) are quite well understood, and various solutions have been proposed [Kha95,DR10,AGR16,Nik15,SEFM15]. Recently, the case of constrained DPPs -when the support is restricted to a combinatorial family B ⊆ 2 [m] -has been studied [NS16,SV16,CDKV16] with machine learning applications in mind, however, very little is known computationally.…”