Building a Fast and Efficient LSM-tree Store by Integrating Local Storage with Cloud Storage

Xu, Peng; Zhao, Nannan; Wan, Jiguang; Liu, Wei; Chen, Shuning; Zhou, Yuanhui; Albahar, Hadeel; Liu, Hanyang; Tan, Zhihu

doi:10.1145/3527452

Cited by 2 publications

(1 citation statement)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this article, we propose RocksMash, a fast and efficient LSM store that uses local storage to store frequently accessed data and metadata while using cloud to hold the rest of the data to achieve Extension of Conference Paper. An earlier version of this article was presented at 2021 IEEE International Conference on Cluster Computing (CLUSTER), 7-10 September, 2021 [58]. In this article, we broaden the design to allow for faster recovery.…”

mentioning

confidence: 99%

Building A Fast and Efficient LSM-tree Store by Integrating Local Storage with Cloud Storage

Zhao

Wan

et al. 2021

2021 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

The explosive growth of modern web-scale applications has made cost-effectiveness a primary design goal for their underlying databases. As a backbone of modern databases, LSM-tree based key-value stores (LSM store) face limited storage options. They are either designed for local storage that is relatively small, expensive, and fast or for cloud storage that offers larger capacities at reduced costs but slower. Designing an LSM store by integrating local storage with cloud storage services is a promising way to balance the cost and performance. However, such design faces challenges such as data reorganization, metadata overhead, and reliability issues. In this article, we propose RocksMash, a fast and efficient LSM store that uses local storage to store frequently accessed data and metadata while using cloud to hold the rest of the data to achieve Extension of Conference Paper. An earlier version of this article was presented at 2021 IEEE International Conference on Cluster Computing (CLUSTER), 7-10 September, 2021 [58]. In this article, we broaden the design to allow for faster recovery. RocksMash proposes the extended WAL to trade faster recovery time for slightly more storage space. RocksMash searches WAL files in reverse chronological order to reduce data to be processed, logically reducing the size of WAL files to be processed, and thus improving scan efficiency. Evaluation results show that the proposed approach improves the recovery performance by up to 10×.

show abstract

mentioning

confidence: 99%

Building A Fast and Efficient LSM-tree Store by Integrating Local Storage with Cloud Storage

Zhao

Wan

et al. 2021

2021 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

show abstract

A Contract-aware and Cost-effective LSM Store for Cloud Storage with Low Latency Spikes

Zhou,

et al. 2024

ACM Trans. Storage

View full text Add to dashboard Cite

Cloud storage is gaining popularity because features such as pay-as-you-go significantly reduce storage costs. However, the community has not sufficiently explored its contract model and latency characteristics. As LSM-Tree-based key-value stores (LSM stores) become the building block for numerous cloud applications, how cloud storage would impact the performance of key-value accesses is vital. This study reveals the significant latency variances of Amazon Elastic Block Store (EBS) under various I/O pressures, which challenges LSM store read performance on cloud storage. To reduce the corresponding tail latency, we propose Calcspar, a contract-aware LSM store for cloud storage, which efficiently addresses the challenges by regulating the rate of I/O requests to cloud storage and absorbing surplus I/O requests with the data cache. We specifically developed a fluctuation-aware cache to lower the high latency brought on by workload fluctuations. Additionally, we build a congestion-aware IOPS allocator to reduce the impact of LSM store internal operations on read latency. We evaluated Calcspar on EBS with different real-world workloads and compared it to the cutting-edge LSM stores. The results show that Calcspar can significantly reduce tail latency while maintaining regular read and write performance, keeping the 99 th percentile latency under 550 μ s and reducing average latency by 66%. In addition, Calcspar has lower write prices and average latency compared to Cloud NoSQL services offered by cloud vendors.

show abstract

Building a Fast and Efficient LSM-tree Store by Integrating Local Storage with Cloud Storage

Cited by 2 publications

References 29 publications

Building A Fast and Efficient LSM-tree Store by Integrating Local Storage with Cloud Storage

Building A Fast and Efficient LSM-tree Store by Integrating Local Storage with Cloud Storage

A Contract-aware and Cost-effective LSM Store for Cloud Storage with Low Latency Spikes

Contact Info

Product

Resources

About