Optimality and scalability in lattice histogram construction

Abstract

The Lattice Histogram is a recently proposed data summarization technique that achieves approximation quality preferable to that of an optimal plain histogram. Like other hierarchical synopsis methods, a lattice histogram (LH) aims to approximate data using a hierarchical structure. Still, this structure is not defined a priori; it consists an unknown, not a given, of the problem. Past work has defined the properties that an LH needs to obey and developed general-purpose approximation algorithms for the construction thereof. Still, two major issues remain unaddressed: First, the construction of an optimal LH for a given error metric is a problem unsolved to date. Second, the proposed algorithms suffer from too high space and time complexities that render their application in real-world settings problematic. In this paper, we address both these questions, focusing on the case that the target error metric is a maximum error metric. Our algorithms treat both the error-bounded LH construction problem, in which the space occupied by an LH is minimized under an error constraint, as well as the classic space-bounded problem. First, we develop a dynamicprogramming scheme that detects an optimal LH under a given maximum-error bound. Second, we propose an efficient, practical, greedy algorithm that solves the same problem with much lower time and space requirements. Then, we show how both our algorithms can be applied to the classic space-bounded problem, aiming at minimizing error under a bound on space. Our experimental study with real-world data sets shows the effectiveness of our methods compared to competing summarization techniques. Moreover, our findings show that our greedy heuristic performs almost as well as the optimal solution in terms of accuracy.

Approximating sliding windows by cyclic tree-like histograms for efficient range queries

“…For instance, c-Tree could be used for clustering [69] and classification tasks, with (hopefully) no appreciable difference in the quality of classification. It would also be interesting to investigate if our bit-saving representation could be applied to other synopses using a hierarchical structure, like the very recent Lattice Histogram [53]. Finally, the high capability of c-Tree to work properly in small space suggests us to study the effectiveness of its hardware implementation, which could be a profitable solution in the field of sensor mining.…”

Efficient Haar + synopsis construction for the maximum absolute error measure

“…A simplified variant of the Haar + tree, called the compact hierarchical histogram (CHH), is introduced in [17,26]. The concept of winning intervals used in our algorithms was previously investigated in [14,17].…”

“…For a set S of synopses with the same size, we define the winning interval of a synopsis s in S as the incoming value interval such that s has the smallest error among all synopses in S. Note that the concept of winning intervals was previously investigated for a hierarchical structure, such as CHH or LH, in [14,17]. Meanwhile, we compute the winning interval of each synopsis in the synopses constructed in each node of a coe cient tree.…”

Fast and effective histogram construction

“…An algorithm that aims to minimize L2 in practice works on the sum-of-squared-errors (SSE) i |di − di| 2 . Previous studies [20,5,23,9,36,24,25,26,21,22] have generalized their results into wider classes of maximum, distributive, Minkowskidistance, and relative-error metrics. Still, the Euclidean error L2 remains an important error metric for several applications, such as database query optimization [16], context recognition [12], and time series mining [4].…”