Optimal histograms for limiting worst-case error propagation in the size of join results

The History of Histograms (abridged)

“…The first results that led towards new types of histograms were derived in an effort to obtain statistics that would be optimal in minimizing/containing the propagation of errors in the size of join results [37]. The basic mathematical tools used were borrowed from majorization theory [55].…”

On Linear-Spline Based Histograms

“…-V-optimal [8,7,9]: Partition data such that β j=1 nj k=1 (f j − f j,k ) 2 is minimized, where β is the number of buckets, n j is the number of entries in the jth bucket, f j is the average frequency of jth bucket, and f j,k is the kth frequency of jth bucket.…”

“…It has been shown [12] that this technique out-performed those "conventional" histogram techniques [7,8,9,13,17]. To compliment the work in [12], in this paper we will propose a novel optimization model for generating linear-spline based histograms.…”

“…The key issues in histogram techniques are: how to partition the original data, what to store in each bucket, and how to estimate the result of an aggregation query for a given histogram. Many histogram techniques have been developed in [7,8,9,10,13,17].…”

Parametric query optimization

“…Histogram H1 in the earlier table is not serial as frequencies 1 and 3 appear in one bucket and frequency 2 appears in the other, while histogram H2 is. Under various optimality criteria, serial histograms have been shown to be optimal for reducing the worst-case and the average error in equality selection and join queries IC93,Ioa93,IP95 . Identifying the optimal histogram among all serial ones takes exponential time in the number of buckets. Moreover, since there is usually no order-correlation between attribute values and their frequencies, storage of serial histograms essentially requires a regular index that will lead to the approximate frequency of every individual attribute value.…”