Speeding Up Queries in Column Stores

Lemke, Christian; Sattler, Kai-Uwe; Faerber, Franz; Zeier, Alexander

doi:10.1007/978-3-642-15105-7_10

Cited by 38 publications

(39 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Compression techniques are typically employed to reduce the amount of consumed memory, and potentially speed up processing. The simplest and most common compression is dictionary encoding [24]. In Figure 3, we show the data structures that compose a column in a generic column-store (naming can be different), along with an optional index.…”

Section: Main-memory Column-storesmentioning

confidence: 99%

Scaling up concurrent main-memory column-store scans

et al. 2015

View full text Add to dashboard Cite

Main-memory column-stores are called to efficiently use modern non-uniform memory access (NUMA) architectures to service concurrent clients on big data. The efficient usage of NUMA architectures depends on the data placement and scheduling strategy of the column-store. Most column-stores choose a static strategy that involves partitioning all data across the NUMA architecture, and employing a stealingbased task scheduler. In this paper, we implement different strategies for data placement and task scheduling for the case of concurrent scans. We compare these strategies with an extensive sensitivity analysis. Our most significant findings include that unnecessary partitioning can hurt throughput by up to 70%, and that stealing memory-intensive tasks can hurt throughput by up to 58%. Based on our analysis, we envision a design that adapts the data placement and task scheduling strategy to the workload.

show abstract

Section: Main-memory Column-storesmentioning

confidence: 99%

Scaling up concurrent main-memory column-store scans

et al. 2015

View full text Add to dashboard Cite

show abstract

“…The standard query operators are directly applied to the compressed data structures. Lemke et al [11] proposed scan and aggregation operators that are designed to operate on top of compressed data. However, it is not known how joins are processed for independently compressed columns.…”

Section: Related Workmentioning

confidence: 99%

Joins on encoded and partitioned data

et al. 2014

View full text Add to dashboard Cite

Compression has historically been used to reduce the cost of storage, I/Os from that storage, and buffer pool utilization, at the expense of the CPU required to decompress data every time it is queried. However, significant additional CPU efficiencies can be achieved by deferring decompression as late in query processing as possible and performing query processing operations directly on the still-compressed data. In this paper, we investigate the benefits and challenges of performing joins on compressed (or encoded) data. We demonstrate the benefit of independently optimizing the compression scheme of each join column, even though join predicates relating values from multiple columns may require translation of the encoding of one join column into the encoding of the other. We also show the benefit of compressing "payload" data other than the join columns "on the fly," to minimize the size of hash tables used in the join. By partitioning the domain of each column and defining separate dictionaries for each partition, we can achieve even better overall compression as well as increased flexibility in dealing with new values introduced by updates. Instead of decompressing both join columns participating in a join to resolve their different compression schemes, our system performs a light-weight mapping of only qualifying rows from one of the join columns to the encoding space of the other at run time. Consequently, join predicates can be applied directly on the compressed data. We call this procedure encoding translation. Two alternatives of encoding translation are developed and compared in the paper. We provide a comprehensive evaluation of these alternatives using product implementations of each on the TPC-H data set, and demonstrate that performing joins on encoded and partitioned data achieves both superior performance and excellent compression.

show abstract

“…Die Geschwindigkeit der Operatoren hängt von der für die Materialisierung der Ergebnisse verwendeten Datenstrukturen (Bitvektor oder Vektor von Ganzzahlen), den Eigenschaften der Kompressionstechnik, der Selektivität und vielen anderen Faktoren ab. Wie mögliche Optimierungen für andere Kompressionstechniken und Operatoren (wie zum Beispiel Aggregation) aussehen, wird von [14] Abb. 6 zeigt den Speicherbedarf der eben beschriebenen Datenverteilungen, wobei auch die Größen unter Verwendung des invertierten Indexes enthalten sind.…”

Section: Operatoren Auf Komprimierten Datenunclassified