Designing Distributed Tree-based Index Structures for Fast RDMA-capable Networks

Ziegler, Tobias; Vani, Sumukha Tumkur; Binnig, Carsten; Fonseca, Rodrigo; Kraska, Tim

doi:10.1145/3299869.3300081

Cited by 52 publications

(30 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is thus especially suitable for disaggregated memory and databases [13]. It has been shown to speed up distributed operators such as data shuffling [52], joins [22,23], transactional workloads [24], or indexing [76]. In Farview, we use RDMA to efficiently transfer data through the network so that the query processing thread directly gets the data from the remote buffer pool.…”

Section: Efficient Data Movementmentioning

confidence: 99%

Farview: Disaggregated Memory with Operator Off-loading for Database Engines

Korolija¹,

Koutsoukos²,

Keeton³

et al. 2021

Preprint

View full text Add to dashboard Cite

Cloud deployments disaggregate storage from compute, providing more flexibility to both the storage and compute layers. In this paper, we explore disaggregation by taking it one step further and applying it to memory (DRAM). Disaggregated memory uses network attached DRAM as a way to decouple memory from CPU. In the context of databases, such a design offers significant advantages in terms of making a larger memory capacity available as a central pool to a collection of smaller processing nodes. To explore these possibilities, we have implemented Farview, a disaggregated memory solution for databases, operating as a remote buffer cache with operator offloading capabilities. Farview is implemented as an FPGA-based smart NIC making DRAM available as a disaggregated, network attached memory module capable of performing data processing at line rate over data streams to/from disaggregated memory. Farview supports query offloading using operators such as selection, projection, aggregation, regular expression matching and encryption. In this paper we focus on analytical queries and demonstrate the viability of the idea through an extensive experimental evaluation of Farview under different workloads. Farview is competitive with a local buffer cache solution for all the workloads and outperforms it in a number of cases, proving that a smart disaggregated memory can be a viable alternative for databases deployed in cloud environments.

show abstract

Section: Efficient Data Movementmentioning

confidence: 99%

Farview: Disaggregated Memory with Operator Off-loading for Database Engines

Korolija¹,

Koutsoukos²,

Keeton³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…RPCs can allow the remote server to perform operations that are not possible or easy to do with only READ/WRITE, like moving, resizing and detecting the most requested objects in a distributed key-value store [9] or notifying backup servers to reclaim the space of old logs in order to save new ones [46]. Additionally, as already explained in the previous paragraph, some distributed data structures, like the indexes of key-value stores, might require multiple indirections inside the index to search for the value of the requested key [50]. If the key-value store uses READ to fetch values directly from memory locations, it will have to traverse the index one node at a time until the location of the value is reached.…”

Section: One-sided Rdma Versus Rpc For Data Transfersmentioning

confidence: 99%

Recent Advancements In Distributed System Communications

Argyroulis¹

2021

Preprint

View full text Add to dashboard Cite

Overheads in Operating System kernel network stacks and sockets have been hindering OSes from managing networking operations efficiently for years. Moreover, when building Remote Procedure Calls over TCP, certain TCP features do not match the needs of RPCs, imposing additional overheads. These issues degrade the performance of distributed systems, which rely on fast communications between machines to be able to serve a large number of client requests with low latency and high throughput. The purpose of this literature survey is to look into recent proposals in research literature that aim to overcome these issues. The survey investigates research literature published between 2010-2020, in order to include important advancements during the most recent decade at the time of writing. The proposals found in papers have been categorized into hardware-based and software-based approaches. The former require specialized hardware to offer high communications performance. The latter are implemented in software and don't rely on specialized hardware or require only certain hardware features. Furthermore, the proposals where also classified according to whether they implement kernel bypass, to avoid using the Operating System kernel network stack, or not. The hardware-based approaches examined here are RDMA, programmable Network Interface Controllers (NIC) and System-on-a-Chip (SoC), while the software-based approaches include optimized socket implementations and RPC frameworks, as well as user space networking.

show abstract

“…(1) to implement efficient RDMA-based data structures (e.g., map, tree) are non-trivial, as the address space of each object should be recorded explicitly and this requires more memory footprint [69]; (2) not all data are necessary to be shared and to manage different regions of data in different memory spaces compactly can further improve the data access efficiency for local search and scan ( §4.2).…”

Section: Indexesmentioning

confidence: 99%

G-Tran: Making Distributed Graph Transactions Fast

Chen,

Li,

Zheng

et al. 2021

Preprint

View full text Add to dashboard Cite

Graph transaction processing raises many unique challenges such as random data access due to the irregularity of graph structures, low throughput and high abort rate due to the relatively large read/write sets in graph transactions. To address these challenges, we present G-Tran -an RDMA-enabled distributed in-memory graph database with serializable and snapshot isolation support. First, we propose a graph-native data store to achieve good data locality and fast data access for transactional updates and queries. Second, G-Tran adopts a fully decentralized architecture that leverages RDMA to process distributed transactions with the MPP model, which can achieve high performance by utilizing all computing resources. In addition, we propose a new MV-OCC implementation with two optimizations to address the issue of large read/write sets in graph transactions. Extensive experiments show that G-Tran achieves competitive performance compared with other popular graph databases on benchmark workloads.

show abstract

Designing Distributed Tree-based Index Structures for Fast RDMA-capable Networks

Cited by 52 publications

References 31 publications

Farview: Disaggregated Memory with Operator Off-loading for Database Engines

Farview: Disaggregated Memory with Operator Off-loading for Database Engines

Recent Advancements In Distributed System Communications

G-Tran: Making Distributed Graph Transactions Fast

Contact Info

Product

Resources

About