One-pass wavelet decompositions of data streams

Gilbert, Anna C.; Kotidis, Yannis; Muthukrishnan, S.; Strauss, Martin J.

doi:10.1109/tkde.2003.1198389

Cited by 110 publications

(102 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Wavelets have been used extensively for approximate answering of different query types and/or in different environments: multidimensional aggregate queries (range-sum queries) in OLAP environments [75,76], aggregate and non-aggregate relational queries with computations directly on the stored wavelet coefficients [14], and selection and aggregate queries over streams [28]. As with histograms, there have also been efforts to devise wavelet-based techniques whose approximate query answers are provided with error guarantees [24], as well as to construct and maintain the most important wavelet coefficients dynamically [57].…”

Section: Competitors Of Histogramsmentioning

confidence: 99%

The History of Histograms (abridged)

Ioannidis

2003

Proceedings 2003 VLDB Conference

322

230

View full text Add to dashboard Cite

The history of histograms is long and rich, full of detailed information in every step. It includes the course of histograms in different scientific fields, the successes and failures of histograms in approximating and compressing information, their adoption by industry, and solutions that have been given on a great variety of histogram-related problems. In this paper and in the same spirit of the histogram techniques themselves, we compress their entire history (including their "future history" as currently anticipated) in the given/fixed space budget, mostly recording details for the periods, events, and results with the highest (personally-biased) interest. In a limited set of experiments, the semantic distance between the compressed and the full form of the history was found relatively small! PrehistoryThe word 'histogram' is of Greek origin, as it is a composite of the words 'isto-s' (ιστ os) (= 'mast', also means 'web' but this is not relevant to this discussion) and 'gram-ma' (γραµµα) (= 'something written'). Hence, it should be interpreted as a form of writing consisting of 'masts', i.e., long shapes vertically standing, or something similar. It is not, however, a

show abstract

Section: Competitors Of Histogramsmentioning

confidence: 99%

The History of Histograms (abridged)

Ioannidis

2003

Proceedings 2003 VLDB Conference

322

230

View full text Add to dashboard Cite

show abstract

“…The online update of such structures in a dynamic scenario is also a required property. Sampling [39], hot lists [11,30], wavelets [9,18,24], sketches [10] and histograms [21][22][23] are examples of synopses methods to obtain fast and approximated answers.…”

Section: Data Summarizationmentioning

confidence: 99%

Fading histograms in detecting distribution and concept changes

Sebastião

Gama

Mendonça

2017

Int J Data Sci Anal

View full text Add to dashboard Cite

The remarkable number of real applications under dynamic scenarios is driving a novel ability to generate and gather information. Nowadays, a massive amount of information is generated at a high-speed rate, known as data streams. Moreover, data are collected under evolving environments. Due to memory restrictions, data must be promptly processed and discarded immediately. Therefore, dealing with evolving data streams raises two main questions: (i) how to remember discarded data? and (ii) how to forget outdated data? To maintain an updated representation of the time-evolving data, this paper proposes fading histograms. Regarding the dynamics of nature, changes in data are detected through a windowing scheme that compares data distributions computed by the fading histograms: the adaptive cumulative windows model (ACWM). The online monitoring of the distance between data distributions is evaluated using a dissimilarity measure based on the asymmetry of the Kullback-Leibler divergence. The experimental results support the ability of fading histograms in providing an updated representation of data. Such property works in favor of detecting distribution changes with smaller detection delay time when compared with standard histograms. With respect to the detection of concept changes, the ACWM is compared with 3 known algorithms taken from the literature, using artificial data and using public data sets, presenting better results. Furthermore, we the proposed method was extended for multidimensional and the experiments performed show the ability of the ACWM for detecting distribution changes in these settings.

show abstract

“…The difficulty is that it does not seem easy to compose with the 2-wise independent functions f and g. An example 4-wise independent function would be s(x) = x T Ax+Bx+c for certain random matrices A, B, and c from a special family of second-order Reed-Muller codes [26]. We do not know how to quickly count the number of solutions x to the equation s(x) = d for general matrices A, B and vector c, while we can count such x for heavilystructured A, B, and c coming from a special family based on Reed-Muller codes.…”

Section: Other Approachesmentioning

confidence: 99%

Rectangle-efficient aggregation in spatial data streams

Tirthapura

Woodruff

2012

Proceedings of the 31st ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems

View full text Add to dashboard Cite

We consider the estimation of aggregates over a data stream of multidimensional axis-aligned rectangles. Rectangles are a basic primitive object in spatial databases, and efficient aggregation of rectangles is a fundamental task. The data stream model has emerged as a de facto model for processing massive databases in which the data resides in external memory or the cloud and is streamed through main memory. For a point p, let n(p) denote the sum of the weights of all rectangles in the stream that contain p. We give nearoptimal solutions for basic problems, including (1) the k-th frequency moment F k = points p |n(p)| k , (2) the counting version of stabbing queries, which seeks an estimate of n(p) given p, and (3) identification of heavy-hitters, i.e., points p for which n(p) is large. An important special case of F k is F0, which corresponds to the volume of the union of the rectangles. This is a celebrated problem in computational geometry known as "Klee's measure problem", and our work yields the first solution in the streaming model for dimensions greater than one.

show abstract

One-pass wavelet decompositions of data streams

Cited by 110 publications

References 37 publications

The History of Histograms (abridged)

The History of Histograms (abridged)

Fading histograms in detecting distribution and concept changes

Rectangle-efficient aggregation in spatial data streams

Contact Info

Product

Resources

About