Optimizing database architecture for the new bottleneck: memory access

Manegold, Stefan; Boncz, Peter; Kersten, Martin

doi:10.1007/s007780000031

Cited by 118 publications

(65 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…[9,17,19,26,27]. It maps the value domain of one or more columns to a contiguous integer range [7,9,10,12,18,20]. This mapping replaces column values with unique integer codes and is stored in a separate data structure, the dictionary, which supports two access methods:…”

Section: Dictionaries In Column Storesmentioning

confidence: 99%

See 1 more Smart Citation

Interleaving with coroutines

et al. 2017

View full text Add to dashboard Cite

Index join performance is determined by the efficiency of the lookup operation on the involved index. Although database indexes are highly optimized to leverage processor caches, main memory accesses inevitably increase lookup runtime when the index outsizes the last-level cache; hence, index join performance drops. Still, robust index join performance becomes possible with instruction stream interleaving: given a group of lookups, we can hide cache misses in one lookup with instructions from other lookups by switching among their respective instruction streams upon a cache miss.In this paper, we propose interleaving with coroutines for any type of index join. We showcase our proposal on SAP HANA by implementing binary search and CSB + -tree traversal for an instance of index join related to dictionary compression. Coroutine implementations not only perform similarly to prior interleaving techniques, but also resemble the original code closely, while supporting both interleaved and non-interleaved execution. Thus, we claim that coroutines make interleaving practical for use in real DBMS codebases.

show abstract

Section: Dictionaries In Column Storesmentioning

confidence: 99%

“…For the size range 1MB-2GB, we observe a significant runtime increase when the dictionary outgrows the last level cache (25MB). The increase is caused by main memory accesses (details in Section 2), a known problem for index joins [32] and main memory database systems in general [5,20].…”

Section: Introductionmentioning

confidence: 99%

Interleaving with coroutines

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Hence, different kinds of processing devices use an encoding optimized for the respective device. For example, a CPU encoding has to support effective caching to reduce the memory access cost [41], whereas a GPU encoding has to ensure coalesced memory access of threads to achieve maximal performance [45]. This usually requires transcoding data before or after the data transfer, which is an additional overhead that can break performance.…”

Section: Functional Propertiesmentioning

confidence: 99%

GPU-Accelerated Database Systems: Survey and Open Challenges

Breß

Heimel

Siegmund

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. The vast amount of processing power and memory bandwidth provided by modern graphics cards make them an interesting platform for data-intensive applications. Unsurprisingly, the database research community identified GPUs as effective co-processors for data processing several years ago. In the past years, there were many approaches to make use of GPUs at different levels of a database system. In this paper, we explore the design space of GPU-accelerated database management systems. Based on this survey, we present key properties, important trade-offs and typical challenges of GPU-aware database architectures, and identify major open challenges. Additionally, we survey existing GPU-accelerated DBMSs and classify their architectural properties. Then, we summarize typical optimizations implemented in GPU-accelerated DBMSs. Finally, we propose a reference architecture, indicating how GPU acceleration can be integrated in existing DBMSs.

show abstract

“…MonetDB pioneered this trend [17] , introducing the radix partitioned join algorithm, and pointing out the potential of optimizing for cache and memory. This style of partitioning has been used in many subsequent papers, including [11] and [5] .…”

Section: Related Workmentioning

confidence: 99%

Memory-efficient hash joins

et al. 2014

View full text Add to dashboard Cite

We present new hash tables for joins, and a hash join based on them, that consumes far less memory and is usually faster than recently published in-memory joins. Our hash join is not restricted to outer tables that fit wholly in memory. Key to this hash join is a new concise hash table (CHT), a linear probing hash table that has 100% fill factor, and uses a sparse bitmap with embedded population counts to almost entirely avoid collisions. This bitmap also serves as a Bloom filter for use in multi-table joins.We study the random access characteristics of hash joins, and renew the case for non-partitioned hash joins. We introduce a variant of partitioned joins in which only the build is partitioned, but the probe is not, as this is more efficient for large outer tables than traditional partitioned joins. This also avoids partitioning costs during the probe, while at the same time allowing parallel build without latching overheads. Additionally, we present a variant of CHT, called a concise array table (CAT), that can be used when the key domain is moderately dense. CAT is collision-free and avoids storing join keys in the hash table.We perform a detailed comparison of CHT and CAT against leading in-memory hash joins. Our experiments show that we can reduce the memory usage by one to three orders of magnitude, while also being competitive in performance.

show abstract

Optimizing database architecture for the new bottleneck: memory access

Cited by 118 publications

References 11 publications

Interleaving with coroutines

Interleaving with coroutines

GPU-Accelerated Database Systems: Survey and Open Challenges

Memory-efficient hash joins

Contact Info

Product

Resources

About