Managing Non-Volatile Memory in Database Systems

Renen, Alexander van; Leis, Viktor; Kemper, Alfons; Neumann, Thomas; Hashida, Takushi; Oe, Kazuichi; Doi, Yoshiyasu; Harada, Lilian; Sato, Masaru

doi:10.1145/3183713.3196897

Cited by 90 publications

(49 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For validation, we have integrated all techniques into our storage engine prototype HyMem [33]. Running a write-heavy (100%) YCSB benchmark [10] on a single thread with a DRAM-resident table, Zero logging, Header, and Classic achieves a throughput of 2 M, 1.7 M, and 1.5 M transactions per second, respectively.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Persistent Memory I/O Primitives

Renen

Vogel

Leis

et al. 2019

Proceedings of the 15th International Workshop on Data Management on New Hardware

Self Cite

View full text Add to dashboard Cite

I/O latency and throughput is one of the major performance bottlenecks for disk-based database systems. Upcoming persistent memory (PMem) technologies, like Intel's Optane DC Persistent Memory Modules, promise to bridge the gap between NAND-based flash (SSD) and DRAM, and thus eliminate the I/O bottleneck. In this paper, we provide one of the first performance evaluations of PMem in terms of bandwidth and latency. Based on the results, we develop guidelines for efficient PMem usage and two essential I/O primitives tuned for PMem: log writing and block flushing. ACM Reference Format:

show abstract

Section: Methodsmentioning

confidence: 99%

“…When PMem is used instead, the buffer pool becomes optional. However, as recent work [6,33] has shown, it is still beneficial to use a buffer pool, due to the lower latencies and reduced complexity when working on DRAM compared to PMem. In addition, this architecture is used in most existing disk-based database systems.…”

Section: Page Propagationmentioning

confidence: 99%

Persistent Memory I/O Primitives

Renen

Vogel

Leis

et al. 2019

Proceedings of the 15th International Workshop on Data Management on New Hardware

Self Cite

View full text Add to dashboard Cite

show abstract

“…The last few years have seen a surge in research efforts that investigate how database systems can leverage SCM as persistent main memory. These research efforts can be categorized into: SCM memory management [19,44,45,31], SCM-based persistent data structures [53,6,43,28], optimizing database algorithms for SCM [7,51], new testing frameworks for SCM-based software [26,42], improving the database logging infrastructure [11,52,16], and finally exploring novel, SCMenabled database storage architectures [41,2,22].…”

Section: Scm Drammentioning

confidence: 99%

On the Diversity of Memory and Storage Technologies

Oukid

Lersch

2018

Datenbank Spektrum

View full text Add to dashboard Cite

The last decade has seen tremendous developments in memory and storage technologies, starting with Flash Memory and continuing with the upcoming Storage-Class Memories. Combined with an explosion of data processing, data analytics, and machine learning, this led to a segmentation of the memory and storage market. Consequently, the traditional storage hierarchy, as we know it today, might be replaced by a multitude of storage hierarchies, with potentially different depths, each tailored for specific workloads. In this context, we explore in this "Kurz Erklärt" the state of memory technologies and reflect on their future use with a focus on data management systems.

show abstract

“…Related work address these challenges with solutions that fall in one of three categories (also identified by previous work [16]). First, early proposals leverage NVM as a cheaper alternative to extend DRAM, enabling larger buffer pools [13,17].…”

Section: Introductionmentioning

confidence: 99%

“…Second, persistent data structures [1,4,9,14,18] enable direct fine-grained writes to NVM by issuing out-of-place writes and relying on instruction ordering (SFENCE) and eagerly flushing cache lines (CLFLUSH/CLWB) to make the operation visible (usually by flipping a validity bit). Third, buffer managers were proposed to integrate NVM with the storage hierarchy below DRAM and above SSD [5,15,16]. These approaches access data on DRAM, and therefore they have full control of when data is persisted.…”

Section: Introductionmentioning

confidence: 99%

Persistent Buffer Management with Optimistic Consistency

Lersch

Lehner

Oukid

2019

Proceedings of the 15th International Workshop on Data Management on New Hardware

View full text Add to dashboard Cite

Finding the best way to leverage non-volatile memory (NVM) on modern database systems is still an open problem. The answer is far from trivial since the clear boundary between memory and storage present in most systems seems to be incompatible with the intrinsic memory-storage duality of NVM. Rather than treating NVM either solely as memory or solely as storage, in this work we propose how NVM can be simultaneously used as both in the context of modern database systems. We design a persistent buffer pool on NVM, enabling pages to be directly read/written by the CPU (like memory) while recovering corrupted pages after a failure (like storage). The main benefits of our approach are an easy integration in the existing database architectures, reduced costs (by replacing DRAM with NVM), and faster peak-performance recovery.

show abstract

Managing Non-Volatile Memory in Database Systems

Cited by 90 publications

References 36 publications

Persistent Memory I/O Primitives

Persistent Memory I/O Primitives

On the Diversity of Memory and Storage Technologies

Persistent Buffer Management with Optimistic Consistency

Contact Info

Product

Resources

About