Indexing on modern hardware

Levandoski, Justin J.; Lomet, David; Sengupta, Sudipta; Birka, Adrian; Diaconu, Cristian

doi:10.1145/2588555.2594536

Cited by 14 publications

(8 citation statements)

References 8 publications

(7 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we focus on dynamic hashing and apply Dash to two classic approaches: extendible hashing [12,38] and linear hashing [29,36] . They are both widely used in database and storage systems, such as Oracle ZFS [40] , IBM GPFS [49] , Berkeley DB [3] and SQL Server Hekaton [32] .…”

Section: Dashmentioning

confidence: 99%

See 1 more Smart Citation

Dash: Scalable Hashing on Persistent Memory

Lu,

Hao,

Wang

et al. 2020

Preprint

View full text Add to dashboard Cite

Byte-addressable persistent memory (PM) brings hash tables the potential of low latency, cheap persistence and instant recovery. The recent advent of Intel Optane DC Persistent Memory Modules (DCPMM) further accelerates this trend. Many new hash table designs have been proposed, but most of them were based on emulation and perform sub-optimally on real PM. They were also piece-wise and partial solutions that side-step many important properties, in particular good scalability, high load factor and instant recovery.We present Dash, a holistic approach to building dynamic and scalable hash tables on real PM hardware with all the aforementioned properties. Based on Dash, we adapted two popular dynamic hashing schemes (extendible hashing and linear hashing). On a 24core machine with Intel Optane DCPMM, we show that compared to state-of-the-art, Dash-enabled hash tables can achieve up to ∼3.9× higher performance with up to over 90% load factor and an instant recovery time of 57ms regardless of data size.

show abstract

Section: Dashmentioning

confidence: 99%

“…If overflow count > 0, then we search the stash buckets as the overflow fingerprint area does not have enough space for all overflow records from the bucket. Otherwise, stash access is only needed if there is a matching fingerprint (lines [31][32][33][34][35].…”

Section: Record Operationsmentioning

confidence: 99%

Dash: Scalable Hashing on Persistent Memory

Lu,

Hao,

Wang

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…It is designed for high-concurrency OLTP, with utilization of lock-free or latch-free data structures (e.g., latch-free hash and range indexes) [86], [230], [231], and an optimistic MVCC technique [107]. It also incorporates a framework, called Siberia [134], [232], [233], to manage hot and cold data differently, equipping it with the capacity to handle Big Data both economically and efficiently.…”

Section: Hekatonmentioning

confidence: 99%

“…Hekaton proposes a latch-free B-tree index, called Bw-tree [86], [230], which uses delta updates to make state changes, based on atomic compare-and-swap (CAS) instructions and an elastic virtual page 10 management subsystem-LLAMA [231]. LLAMA provides a 8.…”

Section: Hekatonmentioning

confidence: 99%

In-Memory Big Data Management and Processing: A Survey

Zhang

Chen

Ooi

et al. 2015

IEEE Trans. Knowl. Data Eng.

355

156

View full text Add to dashboard Cite

Growing main memory capacity has fueled the development of in-memory big data management and processing. By eliminating disk I/O bottleneck, it is now possible to support interactive data analytics. However, in-memory systems are much more sensitive to other sources of overhead that do not matter in traditional I/O-bounded disk-based systems. Some issues such as fault-tolerance and consistency are also more challenging to handle in in-memory environment. We are witnessing a revolution in the design of database systems that exploits main memory as its data storage layer. Many of these researches have focused along several dimensions: modern CPU and memory hierarchy utilization, time/space efficiency, parallelism, and concurrency control. In this survey, we aim to provide a thorough review of a wide range of in-memory data management and processing proposals and systems, including both data storage systems and data processing frameworks. We also give a comprehensive presentation of important technology in memory management, and some key factors that need to be considered in order to achieve efficient in-memory data management and processing.

show abstract

“…FAST [21] uses the same model, and achieves twice query throughput as PALM at a cost of not being able to make updates to the index tree. The Bw-tree [26], developed for Hekaton [15,25], is another latch-free B-tree which manages its memory layout in a log-structured manner and is well-suited for flash solid state disks (SSDs) where random writes are costlier than sequential ones.…”

Section: Latch and Parallelizabilitymentioning

confidence: 99%

PI : a Parallel in-memory skip list based Index

Xie¹,

Cai²,

Jagadish³

et al. 2016

Preprint

View full text Add to dashboard Cite

Due to the coarse granularity of data accesses and the heavy use of latches, indices in the B-tree family are not efficient for in-memory databases, especially in the context of today's multi-core architecture.In this paper, we present PI, a Parallel in-memory skip list based Index that lends itself naturally to the parallel and concurrent environment, particularly with non-uniform memory access. In PI, incoming queries are collected, and disjointly distributed among multiple threads for processing to avoid the use of latches. For each query, PI traverses the index in a Breadth-First-Search (BFS) manner to find the list node with the matching key, exploiting SIMD processing to speed up the search process. In order for query processing to be latch-free, PI employs a light-weight communication protocol that enables threads to re-distribute the query workload among themselves such that each list node that will be modified as a result of query processing will be accessed by exactly one thread. We conducted extensive experiments, and the results show that PI can be up to three times as fast as the Masstree, a state-of-the-art B-tree based index.

show abstract

Indexing on modern hardware

Cited by 14 publications

References 8 publications

Dash: Scalable Hashing on Persistent Memory

Dash: Scalable Hashing on Persistent Memory

In-Memory Big Data Management and Processing: A Survey

PI : a Parallel in-memory skip list based Index

Contact Info

Product

Resources

About