The rapid pace of innovation in non-volatile memory technologies such as 3D Xpoint [1], NVDIMM [2], and zSSD [3] is set to transform how we build, deploy, and manage data service platforms. In particular, the emergence of a byte-addressable and persistent type of memory changes the landscape of the current storage architecture, consolidating different functionalities of memory and storage into a single layer [4]. To take full advantage of this advanced technology, this letter presents a crash-resilient skip list (CRAST) which serves as an in-memory data management module in a key-value store to support crash-consistency from a system failure when running on non-volatile memory. By maintaining the persistent in-memory data in a consistent manner, the proposed skip list provides strong reliability and high performance simultaneously in modern data service platforms. We demonstrate the efficacy of CRAST by implementing its prototype in LevelDB. We experimentally show that CRAST provides excellent performance across various workloads, compared to the original key-value store without any compromise on reliability.
In this paper, we examine the design tradeoffs of existing in-memory data structures of a state-of-the-art key-value store. We observe that no data structures provide both fast point-accesses and consistent ranged- retrievals, and naive amalgamations of existing structures fail to get the best of both worlds. Furthermore, our experiments reveal a performance anomaly when increasing the memory size: as more key-value pairs are maintained in memory, the shortcomings of the data structures exacerbate. To address the above problems, we present TeksDB, a fast and consistent key-value store with a novel in-memory data structure, which effciently handles both point- and ranged- accesses at a modest increase in memory footprint. Our evaluation demonstrates that TeksDB outperforms RocksDB by 3.6×, 9×, and 4.5× for get, scan, and range_query, respectively. The effectiveness of TeksDB extends to real-world workloads, achieving up to 3.3× speedup for YCSB.
Key-value stores (KVS) are now an integral part of modern dataintensive systems. thanks to its simplicity, scalability, and efficiency over traditional database systems. Databases such as MySQL employ KVS (in this case, RocksDB as their backend storage instead of conventional database engines because KVS is less affected by data fragmentation. OLAP and big data analytics applications often use KVS as the backend store to make use of the efficient range_query operations of LSM(Log Structured Merge Tree)-based KVS. Furthermore, the increasing use of KVS as backend stores in distributed systems diversifies the workload for each local KVS, making it necessary to design a highly-concurrent data store that simultaneously processes mixed operations. The underlying technology trends also put immense pressure on the design of KVS. Nowadays, many core servers with hundred GBs of memory are common in production environments, and this enhanced density of hardware technologies (both in terms of the number of cores and memory capacity) leads to more data items residing in memory. This phenomenon places greater emphasis on the efficiency of the in-memory data structure in KVS, as more data accesses are serviced from its memory component, which is often called the memtable. The current memtable structure of KVS, however, generally assumes a limited memory capacity that only serves as a temporary write buffer, and the use of single-threaded writes or unscalable data structures for the memtable fails to fulfill this external demand.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.