Block CUR: Decomposing Matrices Using Groups of Columns

Abstract

A common problem in large-scale data analysis is to approximate a matrix using a combination of specifically sampled rows and columns, known as CUR decomposition. Unfortunately, in many real-world environments, the ability to sample specific individual rows or columns of the matrix is limited by either system constraints or cost. In this paper, we consider matrix approximation by sampling predefined blocks of columns (or rows) from the matrix. We present an algorithm for sampling useful column blocks and provide novel guarantees for the quality of the approximation. This algorithm has application in problems as diverse as biometric data analysis to distributed computing. We demonstrate the effectiveness of the proposed algorithms for computing the Block CUR decomposition of large matrices in a distributed setting with multiple nodes in a compute cluster, where such blocks correspond to columns (or rows) of the matrix stored on the same node, which can be retrieved with much less overhead than retrieving individual columns stored across different nodes. In the biometric setting, the rows correspond to different users and columns correspond to users' biometric reaction to external stimuli, e.g., watching video content, at a particular time instant. There is significant cost in acquiring each user's reaction to lengthy content so we sample a few important scenes to approximate the biometric response. An individual time sample in this use case cannot be queried in isolation due to the lack of context that caused that biometric reaction. Instead, collections of time segments (i.e., blocks) must be presented to the user. The practical application of these algorithms is shown via experimental results using real-world user biometric data from a content testing environment.

Orthonormal Sketches for Secure Coded Regression

“…Leverage scores induce a sampling distribution which has proven to be useful in linear regression [3], [67], [73], [75] and GC [45]. From these lemmas, we deduce that the leverage scores of ĤDA are close to being uniform, implying that the block leverage scores [45], [65] are also uniform, which is precisely what Lemma 9 states. Lemma 8 is a variant of the Flattening Lemma [27], [73], a key result to Hadamard based sketching algorithms, which justifies uniform sampling.…”

“…This is precisely what allows us to sample blocks for the construction of S Π ; and in the distributed approach the computations, uniformly at random. Recall that the leverage scores of Ũ := ΠU are ℓ i := ∥ Ũ(i) ∥ 2 2 for i ∈ N N , and the block leverage scores [45], [65] are defined as lι := ∥ Ũ(Kι) ∥ 2 F = j∈Kι ℓ j for all ι ∈ N K . A lot of work has been done regarding ℓ 2 -s.e.…”

Weighted Gradient Coding with Leverage Score Sampling

“…Furthermore, existing block sampling algorithms can also benefit from the proposed expansion networks, e.g. CR-multiplication [28] and CU R decomposition [27]. For instance, a coded matrix multiplication algorithm of minimum variance can been designed, where the sampling distribution proposed in [28] is used to determine the replication numbers of the expansion network.…”

“…The normalized block leverage scores, introduced independently in [27], [31], are the sum of the normalized leverage scores of the subset of rows constituting the block. Analogous to (4), considering the partitioning of D according to K {K} , the normalized block leverage scores of A are defined as…”

“…This framework accommodates a central class of sketching algorithms, that of importance (block) sampling algorithms (e.g. CU R decomposition [27], CR-multiplication [28]). Coded computing is a novel computing paradigm that utilizes coding theory to effectively inject and leverage data and computation redundancy to mitigate errors and slow or non-responsive servers; known as stragglers, among other fundamental bottlenecks, in large-scale distributed computing.…”