Interleaving with coroutines

Abstract

Index join performance is determined by the efficiency of the lookup operation on the involved index. Although database indexes are highly optimized to leverage processor caches, main memory accesses inevitably increase lookup runtime when the index outsizes the last-level cache; hence, index join performance drops. Still, robust index join performance becomes possible with instruction stream interleaving: given a group of lookups, we can hide cache misses in one lookup with instructions from other lookups by switching among their respective instruction streams upon a cache miss.In this paper, we propose interleaving with coroutines for any type of index join. We showcase our proposal on SAP HANA by implementing binary search and CSB + -tree traversal for an instance of index join related to dictionary compression. Coroutine implementations not only perform similarly to prior interleaving techniques, but also resemble the original code closely, while supporting both interleaved and non-interleaved execution. Thus, we claim that coroutines make interleaving practical for use in real DBMS codebases.

Bridging the Latency Gap between NVM and DRAM for Latency-bound Operations

“…Traditionally, interleaving has implied extensive code rewrites with techniques like group prefetching [11] and asynchronous memory access chaining [24], and has thus been avoided in production environments in favor of maintainability. Recent proposals [21,23,31,32] avoid the prohibitive code rewrites by encoding the independent tasks as coroutines, i.e., functions that suspend their execution at specified points and later resume from where they left off. Listing 1 hints the changes required to enable interleaved execution through an example depicting a binary search implemented as a C++20 coroutine [6].…”

“…Note these suspensions are an essential part of converting call chain nodes from ordinary functions to coroutines, but have no direct connection to latency hiding. When the task<int> of binary_search suspends on a cache miss, execution control returns to for_each; for_each has a round-robin coroutine scheduler [31] managing a group of G coroutines running interleaved, where G is large enough to hide the memory latency [32]. After the evaluation of the co_await expression, the returned position is used to check if array[position] equals to value, in which case position is added to positions.…”

“…We use the binary search microbenchmark described in [31], as representative of a simple index join. We use two implementations that look for a list of values in a sorted array of 32-bit signed (int32_t) integers.…”

“…In the aforementioned cases of index joins and tuple reconstruction, the respective index and column lookups do not depend on each other, allowing the processor to "context switch" among lookups upon cache misses and continue executing instructions instead of waiting for data to be fetched from main memory. As prior work [21,31,32] on join-like operations demonstrates, this form of interleaved execution can be implemented in a practical manner using coroutines, i.e., functions that can suspend their execution and be resumed at a later point. Our work adds NVM latency to the performance equation, focusing on the following questions: (a) What is the performance difference between NVM and DRAM for latency-bound operations?…”

“…(b) How much does interleaved execution reduce this difference? We address both questions with the binary search microbenchmark of [31], as well as the end-to-end execution of two queries: a query with an IN-predicate and one with a simple SELECT * statement, representing index joins and tuple reconstruction respectively.…”