iLSM-SSD: An Intelligent LSM-Tree Based Key-Value SSD for Data Analytics

Lee, Chang-Gyu; Kang, Hyeongu; Park, Donggyu; Park, Sung-Yong; Kim, Young-Jae; Noh, Jungki; Chung, Woosuk; Park, Kyoung

doi:10.1109/mascots.2019.00048

Cited by 13 publications

(4 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because of its increasing popularity across a variety of systems, many LSM-tree variants have been proposed [4,24,48]. Recently LSM-tree has also been used in some implementations of KV-SSDs like LightStore [11] and iLSM-SSD [29].…”

Section: Lsm-tree-based Kv-ssdmentioning

confidence: 99%

See 1 more Smart Citation

Design of LSM-tree-based Key-value SSDs with Bounded Tails

Bae

Chung

et al. 2021

ACM Trans. Storage

View full text Add to dashboard Cite

Key-value store based on a log-structured merge-tree (LSM-tree) is preferable to hash-based key-value store, because an LSM-tree can support a wider variety of operations and show better performance, especially for writes. However, LSM-tree is difficult to implement in the resource constrained environment of a key-value SSD (KV-SSD), and, consequently, KV-SSDs typically use hash-based schemes. We present PinK , a design and implementation of an LSM-tree-based KV-SSD, which compared to a hash-based KV-SSD, reduces 99th percentile tail latency by 73%, improves average read latency by 42%, and shows 37% higher throughput. The key idea in improving the performance of an LSM-tree in a resource constrained environment is to avoid the use of Bloom filters and instead, use a small amount of DRAM to keep/pin the top levels of the LSM-tree. We also find that PinK is able to provide a flexible design space for a wide range of KV workloads by leveraging the read-write tradeoff in LSM-trees.

show abstract

Section: Lsm-tree-based Kv-ssdmentioning

confidence: 99%

“…We have used LightStore [11] as the baseline for an LSM-tree-based KV-SSD that separates keys and values to speed up writes and uses Bloom filters to speed up reads. We expect iLSM-SSD [29] to exhibit similar read and write performance as LightStore, because it follows same concepts.…”

Section: Challenges In Implementing Lsm-tree In a Kv-ssdmentioning

confidence: 99%

Design of LSM-tree-based Key-value SSDs with Bounded Tails

Bae

Chung

et al. 2021

ACM Trans. Storage

View full text Add to dashboard Cite

show abstract

“…To address this issue, various methodologies have been introduced to eliminate the file system and operating system components in the I/O path of KVS [8][9][10][11][12]. One prominent approach is the Intel Storage Performance Development Kit (SPDK) [13].…”

Section: Introductionmentioning

confidence: 99%

OctoFAS: A Two-Level Fair Scheduler That Increases Fairness in Network-Based Key-Value Storage

Park,

Park

et al. 2024

Electronics

Self Cite

View full text Add to dashboard Cite

We identified a fairness problem in a network-based key-value storage system using Intel Storage Performance Development Kit (SPDK) in a multitenant environment. In such an environment, each tenant’s I/O service rate is not fairly guaranteed compared to that of other tenants. To address the fairness problem, we propose OctoFAS, a two-level fair scheduler designed to improve overall throughput and fairness among tenants. The two-level scheduler of OctoFAS consists of (i) inter-core scheduling and (ii) intra-core scheduling. Through inter-core scheduling, OctoFAS addresses the load imbalance problem that is inherent in SPDK on the storage server by dynamically migrating I/O requests from overloaded cores to underloaded cores, thereby increasing overall throughput. Intra-core scheduling prioritizes handling requests from starving tenants over well-fed tenants within core-specific event queues to ensure fair I/O services among multiple tenants. OctoFAS is deployed on a Linux cluster with SPDK. Through extensive evaluations, we found that OctoFAS ensures that the total system throughput remains high and balanced, while enhancing fairness by approximately 10% compared to the baseline, when both scheduling levels operate in a hybrid fashion.

show abstract

“…This is accomplished by reusing the controller and memory components of a device to perform data computation [7,46,52,76]. Today, we call such a device a computational storage device (CSD) [6], and we have seen people leveraging these devices for compression [55,106], query acceleration [36,44,92], data analytics [19,29,50,83,87], the implementation of an LSM-Tree [47,54,96], and the implementation of a key-value store [49,51,65]. In addition to on-device computation, another common form of computational storage involves the use of dedicated storage servers, or computational storage processors (CSPs) [6], to accelerate data processing.…”

mentioning

confidence: 99%

Streaming Data Reorganization at Scale with DeltaFS Indexed Massive Directories

et al. 2020

View full text Add to dashboard Cite

Complex storage stacks providing data compression, indexing, and analytics help leverage the massive amounts of data generated today to derive insights. It is challenging to perform this computation, however, while fully utilizing the underlying storage media. This is because, while storage servers with large core counts are widely available, single-core performance and memory bandwidth per core grow slower than the core count per die. Computational storage offers a promising solution to this problem by utilizing dedicated compute resources along the storage processing path. We present DeltaFS Indexed Massive Directories (IMDs), a new approach to computational storage. DeltaFS IMDs harvest available (i.e., not dedicated) compute, memory, and network resources on the compute nodes of an application to perform computation on data. We demonstrate the efficiency of DeltaFS IMDs by using them to dynamically reorganize the output of a real-world simulation application across 131,072 CPU cores. DeltaFS IMDs speed up reads by 1,740× while only slightly slowing down the writing of data during simulation I/O for in situ data processing.

show abstract

iLSM-SSD: An Intelligent LSM-Tree Based Key-Value SSD for Data Analytics

Cited by 13 publications

References 14 publications

Design of LSM-tree-based Key-value SSDs with Bounded Tails

Design of LSM-tree-based Key-value SSDs with Bounded Tails

OctoFAS: A Two-Level Fair Scheduler That Increases Fairness in Network-Based Key-Value Storage

Streaming Data Reorganization at Scale with DeltaFS Indexed Massive Directories

Contact Info

Product

Resources

About