Chucky: A Succinct Cuckoo Filter for LSM-Tree

Dayan, Niv; Twitto, Moshe

doi:10.1145/3448016.3457273

Cited by 30 publications

(8 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [8] the authors make use of cuckoo filters by replacing the multiple bloom filters that are being used as a single main cuckoo filter in the LSM tree. This approach achieves the mentioned solution by using bits that should be used by the fingerprint to map data into auxiliary addresses in the LSM tree.…”

Section: Novel Cuckoo Filters Related Researchmentioning

confidence: 99%

See 1 more Smart Citation

LSM Tree Optimization Through The Implementation of Cuckoo Filters

Valverde,

Kim,

Kim

et al. 2023

Preprint

View full text Add to dashboard Cite

This study aims to develop an LSM tree that uses two different kinds of probabilistic data structures (PDS). These two data structures are the Bloom Filters and the state-of-the-art Cuckoo Filter released in 2014 by Fan. Cuckoo filters are a perfect choice for saving space and also for deleting keys that are not needed in the algorithm so the filters are synchronized with the keys of all the SSTables. The implementation in this study of Cuckoo filters showed three main advantages (1) Keys that are deleted in the SSTables are immediately reflected in the Cuckoo Filters (2) When SSTables are merged, Cuckoo Filters are also merged by measuring its load factor for availability (3) The performance of Cuckoo Filters compared to Bloom filters after deleting and query shows better performance despite that cuckoo filters are also being updated. This approach is expected to reduce storage space and the amount of time for queries and deletions on the overall data structure. This new optimized LSM tree is expected to be implemented in HBase as future work to check its performance with a real-world database for more empirical results.

show abstract

Section: Novel Cuckoo Filters Related Researchmentioning

confidence: 99%

“…To address this issue, instead of using bits that are designated to the fingerprints, encoding is used to mitigate the high false positive rate. However, one of the main challenges that [8] have is that the LSM tree can grow in size over time. Therefore, the higher the level, the more bits are needed.…”

Section: Novel Cuckoo Filters Related Researchmentioning

confidence: 99%

LSM Tree Optimization Through The Implementation of Cuckoo Filters

Valverde,

Kim,

Kim

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Bloom filters provide a membership query of a search key against each SSTable which answers the presence of the given key without traversing the SSTable. We have seen good progress in this area of optimization [18], [24]- [27], all the efforts made to narrow the search space on a key basis.…”

Section: Background and Related Workmentioning

confidence: 99%

Hybrid Transactional/Analytical Processing Amplifies IO in LSM-Trees

et al. 2022

View full text Add to dashboard Cite

The log-structured merge tree (LSM-tree) has become an essential component in many key-value systems and expanded its scope to full-fledged database engines (e.g., MyRocks). In the database landscape, vendors face growing customer demands for real-time analytic solutions to handle hybrid transactional/analytical processing (HTAP) workloads that pose significant challenges. Among the challenges is IO amplification that drives system designers to rethink write-optimized engines to survive HTAP loads. This paper follows the same philosophy, reexamines LSM-trees used for database systems, and rethinks IO amplification under HTAP loads to shed some light on practical remedies for upcoming challenges. We propose two practical techniques to alleviate IO amplification: 1) aligned compaction for reducing write amplification, 2) snapshot filters for reducing read amplification. The two techniques are lightweight (i.e., near-zero resource consumption) and are compatible with state-of-the-art methods. We integrated our techniques into RocksDB and demonstrated that the modified RocksDB exhibits reduced IO amplification under HTAP workloads with negligible resource consumption.

show abstract

“…Bloom-Filters are well-known and with many variants [1,5,28,46] covering different aspects: counting [4,18,41]; compressibility [31]; SIMD vectorization [25,37]; partial deletes [40]; efficient hashing [15,23]; and data locality and novel hardware [6,14,25,27,39]. Recently, there have been numerous novel proposals [11,12,21,35,47], all of which are point-filters with different properties. Pioneered by [24,32], the concept of learned BFs, leads to interesting applications [22,26,48] and is a future direction for bloomRF.…”

Section: Related Workmentioning

confidence: 99%

bloomRF: On Performing Range-Queries in Bloom-Filters with Piecewise-Monotone Hash Functions and Prefix Hashing

Mößner¹,

Riegger²,

Bernhardt³

et al. 2022

Preprint

View full text Add to dashboard Cite

We introduce bloomRF as a unified method for approximate membership testing that supports both pointand range-queries. As a first core idea, bloomRF introduces novel prefix hashing to efficiently encode range information in the hash-code of the key itself. As a second key concept, bloomRF proposes novel piecewisemonotone hash-functions that preserve local order and support fast range-lookups with fewer memory accesses. bloomRF has nearoptimal space complexity and constant query complexity. Although, bloomRF is designed for integer domains, it supports floating-points, and can serve as a multi-attribute filter. The evaluation in RocksDB and in a standalone library shows that it is more efficient and outperforms existing point-range-filters by up to 4× across a range of settings and distributions, while keeping the false-positive rate low.

show abstract

Chucky: A Succinct Cuckoo Filter for LSM-Tree

Cited by 30 publications

References 45 publications

LSM Tree Optimization Through The Implementation of Cuckoo Filters

LSM Tree Optimization Through The Implementation of Cuckoo Filters

Hybrid Transactional/Analytical Processing Amplifies IO in LSM-Trees

bloomRF: On Performing Range-Queries in Bloom-Filters with Piecewise-Monotone Hash Functions and Prefix Hashing

Contact Info

Product

Resources

About