A PTAS for ℓ_p-Low Rank Approximation

Abstract

A number of recent works have studied algorithms for entrywise p-low rank approximation, namely algorithms which given an n×d matrix A (with n ≥ d), output a rank-k matrix B minimizing A − B p p = i,j |Ai,j − Bi,j| p when p > 0; and A − B 0 = i,j [Ai,j = Bi,j] for p = 0, where [·] is the Iverson bracket, that is, A − B 0 denotes the number of entries (i, j) for which Ai,j = Bi,j. For p = 1, this is often considered more robust than the SVD, while for p = 0 this corresponds to minimizing the number of disagreements, or robust PCA. This problem is known to be NP-hard for p ∈ {0, 1}, already for k = 1, and while there are polynomial time approximation algorithms, their approximation factor is at best poly(k). It was left open if there was a polynomial-time approximation scheme (PTAS) for p-approximation for any p ≥ 0. We show the following:1. On the algorithmic side, for p ∈ (0, 2), we give the first n poly(k/ε) time (1 + ε)approximation algorithm. For p = 0, there are various problem formulations, a common one being the binary setting in which A ∈ {0, 1} n×d and B = U · V , where U ∈ {0, 1} n×k and V ∈ {0, 1} k×d . There are also various notions of multiplication U · V , such as a matrix product over the reals, over a finite field, or over a Boolean semiring. We give the first almost-linear time approximation scheme for what we call the Generalized Binary 0-Rank-k problem, for which these variants are special cases. Our algorithm computes (1 + ε)-approximation in time (1/ε) 2 O(k) /ε 2 · nd 1+o(1) , where o(1) hides a factor (log log d) 1.1 / log d. In addition, for the case of finite fields of constant size, we obtain an alternate PTAS running in time n · d poly(k/ε) . Definition 2. (Generalized Binary 0 -Rank-k) Given a matrix A ∈ {0, 1} n×d with n ≥ d, an integer k, and an inner product function ., . :Our first result for p = 0 is as follows.Theorem 2 (PTAS for p = 0). For any ε ∈ (0, 1 2 ), there is a (1+ε)-approximation algorithm for the Generalized Binary 0 -Rank-k problem running in time (1/ε) 2 O(k) /ε 2 · nd 1+o(1) and succeeds with constant probability 1 , where o(1) hides a factor (log log d)Hence, we obtain the first almost-linear time approximation scheme for the Generalized Binary 0 -Rank-k problem, for any constant k. In particular, this yields the first polynomial time (1+ε)-approximation for constant k for 0 -low rank approximation of binary matrices when the underlying field is F 2 or the Boolean semiring. Even for k = 1, no PTAS was known before.Theorem 2 is doubly-exponential in k, and we show below that this is necessary for any approximation algorithm for Generalized Binary 0 -Rank-k. However, in the special case when the base field is F 2 , or more generally F q and A, U, and V have entries belonging to F q , it is possible to obtain an algorithm running in time n·d poly(k/ε) , which is an improvement for certain super-constant values of k and ε. We formally define the problem and state our result next. Definition 3. (Entrywise 0 -Rank-k Approximation over F q ) Given an n × d matrix A with e...

Approximation Schemes for Low-rank Binary Matrix Approximation Problems

“…Let us note that independently Ban et al [6] obtained a very similar algorithmic result for low-rank binary approximation. Their algorithm runs in time 1 2 O(r) / 2 n· m · log 2 r n. Moreover, they also obtained a lower bound of 2 2 δr for a constant δ under Small Set Expansion Hypothesis and Exponential Time Hypothesis.…”

“…In particular, the ideas from the algorithm for k-Means Clustering of Kumar et al [26] form the starting point of our algorithm for Binary Constrained Clustering. 6…”

“…Let π be a linear order of S according to the non-decreasing distance d H (v, C 1 ), where v ∈ S. 6 Let S be the last |S| 2 vectors in the order π.…”

Recent Developments in Boolean Matrix Factorization

“…they require superlinear running time even when k is a fixed constant. Ban et al (2019) obtained a similar algorithm with slightly worse running time, but their algorithm extends to any finite field.…”

“…Bhattacharya et al (2019) extended ideas of Fomin et al (2019) and Ban et al (2019) to obtain a 4-pass streaming algorithm which computes a (1 + ε)-approximate BMF. Their algorithm never stores more than 2Õ (2 k /ε 2 ) • (lg n) 2k rows of the matrix and it has running time 2Õ (2 k /ε 2 ) • (lg n) 2k • mn.…”