Proportional Volume Sampling and Approximation Algorithms for A-Optimal Design

Abstract

We study the A-optimal design problem where we are given vectors v 1 , . . . , v n ∈ R d , an integer k ≥ d, and the goal is to select a set S of k vectors that minimizes the trace ofthe problem is an instance of optimal design of experiments in statistics (Pukelsheim (2006)) where each vector corresponds to a linear measurement of an unknown vector and the goal is to pick k of them that minimize the average variance of the error in the maximum likelihood estimate of the vector being measured. The problem also finds applications in sensor placement in wireless networks (Joshi and Boyd (2009)), sparse least squares regression (Boutsidis et al. (2011)), feature selection for k-means clustering (Boutsidis and Magdon-Ismail (2013)), and matrix approximation Mattheij (2007, 2011); Avron and Boutsidis (2013)). In this paper, we introduce proportional volume sampling to obtain improved approximation algorithms for A-optimal design. Given a matrix, proportional volume sampling involves picking a set of columns S of size k with probability proportional to µ(S) times det( i∈S v i v i ) for some measure µ. Our main result is to show the approximability of the A-optimal design problem can be reduced to approximate independence properties of the measure µ. We appeal to hard-core distributions as candidate distributions µ that allow us to obtain improved approximation algorithms for the A-optimal design. Our results include a d-approximation when k = d, an (1 + )-approximation when k = Ω d + 1 2 log 1 and k k−d+1approximation when repetitions of vectors are allowed in the solution. We also consider generalization of the problem for k ≤ d and obtain a k-approximation.We also show that the proportional volume sampling algorithm gives approximation algorithms for other optimal design objectives (such as D-optimal design Singh and Xie (2018) and generalized ratio objective Mariet and Sra (2017)) matching or improving previous best known results. Interestingly, we show that a similar guarantee cannot be obtained for the E-optimal design problem. We also show that the A-optimal design problem is NP-hard to approximate within a fixed constant when k = d.

Determinantal Point Processes in Randomized Numerical Linear Algebra

“…A number of optimality criteria (such as A-optimality, which uses mean squared error of the least squares estimator) have been considered for selecting the subsets. DPP subset selection has been shown to provide useful guarantees for some of the most popular criteria (including A-, C-, D-, and V-optimality), leading to new approximation algorithms [7,22]. Stochastic optimization.…”

Modern Subsampling Methods for Large-Scale Least Squares Regression

“…Harman and Rosa (2020) proposed two greedy heuristic algorithms for constructing the optimal subsample with respect to the D-optimality criterion. Wang et al (2017) and Nikolov et al (2019) considered selecting the optimal subsample approximately based on the A-optimality criterion. Allen-Zhu et al (2017) proposed polynomial-time algorithms for numerous classical optimality criteria, such as A-optimality, D-optimality, V -optimality, and others.…”

Design of Experiments with Imputable Feature Data: An Entropy-Based Approach

“…We denote the input data matrix with missing values m ∈ R |G | by X(m) ∈ R n×p . Consistent with related literature [24], [30], [31] in linear design of experiments, we assume X(m) to be of full rank (equal to p) and hence X(m) X(m) is invertible for all m. The SSIO problem is given by min s∈S,m∈M trace X(m) Λ(s)X(m) −1 .…”