Improved Practical Matrix Sketching with Guarantees

Efficient Frequent Directions Algorithm for Sparse Matrices

“…The empirical comparison of FrequentDirections to other matrix sketching techniques is now well-trodden [17,7]. FrequentDirections (and, as we observe, by association SparseFre-quentDirections) has much smaller error than other sketching techniques which operate in a stream.…”

“…Another similar line of work is the CUR factorization [4,10,12,14,27] where methods select c columns and r rows of A to form matrices C ∈ R n×c , R ∈ R r×d and U ∈ R c×r , and constructs the sketch as B = CU R. The only instance of this group that runs in input sparsity time is [4] Random projection techniques These techniques [31,36,35,26] operate data-obliviously and maintain a r×d matrix B = SA using a r×n random matrix S which has the Johnson-Lindenstrauss Transform (JLT) property [28]. Random projection methods work in the streaming model, are computationally efficient, and sufficiently accurate in practice [7]. The state-of-the-art method of this approach is by Clarkson and Woodruff [6] which was later improved slightly in [30].…”

“…Examples of these methods include different version of iterative SVD [19,21,23,5,33]. These, however, do not have theoretical guarantees [7]. The FrequentDirections algorithm [24] is a unique in this group in that it offers strong error guarantees.…”

“…The running time of FrequentDirections and its error analysis are strongly coupled with the properties of the SVD used to perform the DenseShrink step. An analysis of [7] slightly generalized the one in [17]. Let B be the sketch resulting in applying FrequentDirections with a potentially different shrink operation to A.…”

“…Lemma 3.1 (Lemma 3.1 in [7]). Given an input n × d matrix A and an integer parameter , any sketch ×d matrix B which satisfies the three properties above (for some any α ∈ (0, 1] and ∆ > 0), guarantees the following error bounds…”