Probabilistic wavelet synopses

Abstract

Recent work has demonstrated the effectiveness of the wavelet decomposition in reducing large amounts of data to compact sets of wavelet coefficients (termed "wavelet synopses") that can be used to provide fast and reasonably accurate approximate query answers. A major shortcoming of these existing wavelet techniques is that the quality of the approximate answers they provide varies widely, even for identical queries on nearly identical values in distinct parts of the data. As a result, users have no way of knowing whether a particular approximate answer is highlyaccurate or off by many orders of magnitude. In this article, we introduce Probabilistic Wavelet Synopses, the first wavelet-based data reduction technique optimized for guaranteed accuracy of individual approximate answers. Whereas previous approaches rely on deterministic thresholding for selecting the wavelet coefficients to include in the synopsis, our technique is based on a novel, probabilistic thresholding scheme that assigns each coefficient a probability of being included based on its importance to the reconstruction of individual data values, and then flips coins to select the synopsis. We show how our scheme avoids the above pitfalls of deterministic thresholding, providing unbiased, highly accurate answers for individual data values in a data vector. We propose several novel optimization algorithms for tuning our probabilistic thresholding scheme to minimize desired error metrics. Experimental results on real-world and synthetic data sets evaluate these algorithms, and demonstrate the effectiveness of our probabilistic wavelet synopses in providing fast, highly accurate answers with improved quality guarantees.

REHISTRelative Error Histogram Construction Algorithms

“…The sanitary bound c is set to the 10-percent value in the data as in [3,4]. We used our implementations of the probabilistic thresholding scheme [3,4] and the deterministic thresholding scheme [17] as representatives of traditional summarization techniques that considers relative errors as objective function.…”

“…[4,4] 4 We illustrate the decomposition of [2,4] for a given interval tree in Figure 3. The interval [2,4] is decomposed into [2,2] and [3,4]. …”

“…We experimented with four different permutation techniques that were used in [3,4]: NoPerm, Normal, PipeOrgan and Random. Normal permutes the frequencies to resemble a bellshaped normal distribution, with higher frequencies at the center of the domain.…”

Discrete Wavelet Transform and Wavelet Synopses

“…Each data cell in A can be accurately reconstructed by adding up the contributions (with the appropriate signs) of those coefficients whose support regions include the cell. Error-tree structures for d-dimensional Haar coefficients are essentially d-dimensional quadtrees, where each internal node t corresponds to a set of (at most) 2 d − 1 Haar coefficients, and has 2 d children corresponding to the quadrants of the (common) support region of all coefficients in t; furthermore, properties (P1) and (P2) can also be naturally extended to the multi-dimensional case [2,7,8].…”

“…The wavelet transform has a long history of successful applications in signal and image processing [11,12]. Several recent studies have also demonstrated the effectiveness of the wavelet transform (and Haar wavelets, in particular) as a tool for approximate query processing over massive relational tables [2,7,8] and continuous data streams [3,9]. Briefly, the idea is to apply wavelet transform to the input relation to obtain a compact data synopsis that comprises a select small collection of wavelet coefficients.…”