An Information Divergence Estimation over Data Streams

Abstract

In this paper, we consider the setting of large scale distributed systems, in which each node needs to quickly process a huge amount of data received in the form of a stream that may have been tampered with by an adversary. In this situation, a fundamental problem is how to detect and quantify the amount of work performed by the adversary. To address this issue, we have proposed in a prior work, AnKLe, a one pass algorithm for estimating the Kullback-Leibler divergence of an observed stream compared to the expected one. Experimental evaluations have shown that the estimation provided by AnKLe is accurate for different adversarial settings for which the quality of other methods dramatically decreases. In the present paper, considering n as the number of distinct data items in a stream, we show that AnKLe is an (ε, δ)-approximation algorithm with a space complexityÕ( 1 ε + 1 ε 2 ) bits in "most" cases, andÕ( 1 ε + n−ε −1 ε 2 ) otherwise. To the best of our knowledge, an approximation algorithm for estimating the Kullback-Leibler divergence has never been analyzed before.

Sketch ⋆-Metric: Comparing Data Streams via Sketching

“…Subsequently, a lot of attention has been paid to the strongly related notion of the entropy of a stream, and all notions based on entropy (i.e., norm and relative entropy) [11]. These notions are essentially related to the quantification of the amount of randomness of a stream (e.g, [12], [13], [14], [15], [16], [17]). The construction of synopses or sketches of the data stream have been proposed for different applications (e.g, [18], [19], [20]).…”

Reinventing Mobile Community Computing and Communication

“…Most of the research done so far with this approach has focused on computing functions or statistic measures with error ε using poly(1/ε, log n) space where n is the domain size of the data items. These include the computation of the number of different data items in a given stream [31], [32], the frequency moments [33], the most frequent data items [33], [34], the entropy of the stream [35], or the relative entropy between one data stream and the uniform one [36], [37]. A comprehensive survey of these techniques, their advantages and their drawbacks is given in [38].…”

Lightweight Metric Computation for Distributed Massive Data Streams

“…The authors have proposed estimators of the frequency moments F k of a stream, which are important statistical tools that allow to quantify specificities of a data stream. Subsequently, a lot of attention has been devoted to the strongly related notion of the entropy [15] of a stream [16,17,18], and all notions based on entropy as the quantification of the amount of randomness of a stream (e.g, [17,19,20,21]). The construction of synopses or sketches of the data stream have been proposed for different applications (e.g, [22,23,24,25]).…”