Dwarfs in the rearview mirror

Abstract

Online-Analytical Processing (OLAP) has been a field of competing technologies for the past ten years. One of the still unsolved challenges of OLAP is how to provide quick response times on any Terabyte-sized business data problem. Recently, a very clever multi-dimensional index structure termed Dwarf [26] has been proposed offering excellent query response times as well as unmatched index compression rates. The proposed index seems to scale well for both large data sets as well as high dimensions. Motivated by these surprisingly excellent results, we take a look into the rearview mirror. We have re-implemented the Dwarf index from scratch and make three contributions. First, we successfully repeat several of the experiments of the original paper. Second, we substantially correct some of the experimental results reported by the inventors. Some of our results differ by orders of magnitude. To better understand these differences, we provide additional experiments that better explain the behavior of the Dwarf index. Third, we provide missing experiments comparing Dwarf to baseline query processing strategies. This should give practitioners a better guideline to understand for which cases Dwarf indexes could be useful in practice. 1 Using a star schema is already an optimization: it simply replaces all non-fact tables by materialized views.

Brown Dwarf: A fully-distributed, fault-tolerant data warehousing system

“…The other 2 datasets are subsets of the Weather [36] and Forest [7] datasets (10 k tuples each). The forest dataset has 10 dimensions with cardinalities as reported in [11], while the weather dataset has 9 dimensions, corresponding to truncated ocean weather measurements for Sept. 1984. We use the APB query generator to produce 1 k query workloads, both point and aggregate ones.…”

“…While it offers many advantages, like data compression and efficiency in answering aggregate queries, it exhibits certain limitations that prohibit its use as a solution for our motivating problem. Besides the lack of fault-tolerance and decentralization, a Dwarf structure may take up orders of magnitude more space than the original tuples [11]. Our Brown Dwarf system relaxes these storage requirements and enables the computation of much larger cubes.…”

Efficient updates for a shared nothing analytics platform

“…While Dwarf offers many advantages, like data compression and efficiency in answering aggregate queries, it exhibits certain limitations that prohibit its use as a solution for our motivating problem. Besides the lack of decentralization, recent work [6] indicated that, depending on the cube's density, a Dwarf structure may take up orders of magnitude more space than the original tuples. Moreover, updating such a structure is very costly and inefficient, due to the a-priori materialization.…”

Subspace Similarity Search: Efficient k-NN Queries in Arbitrary Subspaces

“…Several studies for research on data structures [1][2][3][4][5] showed that the suitability of the sequential scan [6] compared to methods using partitioning or clustering based data structures is dependent of the characteristics of the data distributions. However, this key message has been neglected in many research contributions [7][8][9][10][11][12]. Thus, it still appears to be well worth noting that nearest neighbor search is meaningful if and only if the nearest neighbor of the arbitrary query object is sufficiently different from its farthest neighbor.…”