Efficient sampling of non-strict turnstile data streams

Abstract

We study the problem of generating a large sample from a data stream S of elements (i, v), where i is a positive integer key, v is an integer equal to the count of key i, and the sample consists of pairs (i, C i ) for C i = (i,v)∈S v. We consider strict turnstile streams and general non-strict turnstile streams, in which C i may be negative. Our sample is useful for approximating both forward and inverse distribution statistics, within an additive error and provable success probability 1 − δ.Our sampling method improves by an order of magnitude the known processing time of each stream element, a crucial factor in data stream applications, thereby providing a feasible solution to the sampling problem. For example, for a sample of size O ( −2 log (1/δ)) in non-strict streams, our solution requires O ((log log(1/ )) 2 + (log log(1/δ)) 2 ) operations per stream element, whereas the best previous solution requires O ( −2 log 2 (1/δ)) evaluations of a fully independent hash function per element. We achieve this improvement by constructing an efficient K -elements recovery structure from which K elements can be extracted with probability 1 − δ. Our structure enables our sampling algorithm to run on distributed systems and extract statistics on the difference between streams.

Parameterized Streaming: Maximal Matching and Vertex Cover

“…Constructions of k-sample recovery mechanisms are known which require spaceÕ(k) and fail only with probability polynomially small in n [5]. We apply this algorithm to the neighborhood of vertices: for each node v, we can maintain an instance of the k-sample recovery sketch (or algorithm) to the vector corresponding to the row of the adjacency matrix for v. Note that as edges are inserted or deleted, we can propagate these to the appropriate k-sample recovery algorithms, without needing knowledge of the full neighborhood of nodes.…”

L p Samplers and Their Applications

“…Some increase in space usage seems inevitable as we have to store § This approach of sampling and looking for duplicates may be folklore; it was described to the authors by T. S. Jayram each individual sample, however for better update times there are alternative solutions with faster processing times than the naive solution. In this direction, Barkay et al [9] have shown an L 0 sampler with O(log s δ ) update time and O(s log s δ ) sample extraction time, at the expense of relaxing the independence requirement of the samples. The extracted samples are guaranteed to be O(log 1 δ )-wise independent which is sufficient for most applications.…”

New Streaming Algorithms for Parameterized Maximal Matching & Beyond

“…Such algorithms have been designed to operate deterministically, and require O(k polylog n) space [1].…”

“…Randomized constructions of k-sample algorithms are known (which use k-sparse recovery algorithms within them), and require O(k polylog n) space [1].…”