Par-BF: A Parallel Partitioned Bloom Filter for Dynamic Data Sets

Abstract

Compared with a hash table, a Bloom Filter (BF) is more space-efficient for supporting fast matching though resulting in a controllable and acceptable false positive probability. The space size of the basic BF is predetermined based on the expected number of elements to be stored. However, we cannot predict the scale of a BF space for dynamic sets. The two existing solutions for supporting dynamic sets, Scalable BF (SBF) and Dynamic BF (DBF), still face some challenges on system performance and memory overhead.This paper presents a new BF for dynamic data sets, called Partitioned BF (Par-BF). Compared with DBF and SBF, the size and the range of the false positive probability can be calculated by a group of formulas to leverage a sweet spot between high-performance and low-overhead. Moreover, Par-BF supports parallel fast matching which can improve the overall throughput. From our trace-driven experimental results, the IOPS of Par-BF outperforms that of DBF and SBF from 6X to 10X, and from 2X to 4X, respectively. Meanwhile, through our proposed garbage collection policy, the memory overhead of Par-BF is less than half of the memory usage of SBF.

Optimizing Bloom Filter: Challenges, Solutions, and Comparisons

“…In scenarios where the bit (or cell) vector is divided into multiple segments, these segments can be accessed simultaneously thereby the thereafter computations can be parallelized. This kind of variants include Space-code BF [88], Dynamic BF [89], Dynamic BF array [90], Par-BF [91], BloomStore [18], Cross-checking BF [72], One hash BF [86], Bloom-1 [92], OMASS [93], Parallel BF [94], etc. The parallelism strategy can efficiently reduce the response time to 1/ξ, where ξ is the number of parallelized instances.…”

“…Multi-class BF [3] Optihash [67] FPF-MBF [43] Retouched BF [75] MPCBF [77] Ternary BF [134] Generalized BF [76] Cross-checking BF [72] Complement BF [73] Yes-no BF [74] VI-CBF [83] FP-CBF [84] Selected Hash [78] [79] Space-code BF [88] Adaptive BF [114] Spectral BF [112] Loglog BF [113] Dynamic BF [89] Variable length signatures [117] Scalable BF [118] DBA [90] Par-BF [91] Weighted BF [119] Popularity conscious BF [120] BloomStore [17] kBF [19] IBLT [21] k-mer BF [131] Spatial BF [17] CBF [36] Deletable BF [133] Distance-sensitive BF [143] Locality-sensitive BF [144] Less hash [85] DLB-BF [96] Combinatorial BF [97] Bloom-1 [92] OMASS [93] Compressed BF [99] Compacted BF [100] Forest-structured BF [106] Stable BF [135] Temporal CBF [136] d-left CBF [101] Memo. Opti.…”

“…Parallelism is helpful to accelerate the query process and thereby increase the throughput of query. Typically, the variants with multiple independent BFs or segments can be accessed in parallel, include DBF [89], DBA [90], Par-BF [91], Scalable BF [118], BloomStore [18], FPF-MBF [43], MPCBF [77], Parallel BF [94], Combinatorial BF [97], OMASS [93], Forest-structured BF [106], DLB-BF [96], and Ultra-fast BF [98]. The parallelism of query process decreases the computation time to 1/k thus improves the query throughput significantly.…”

“…IN CONTRAST, WE CONSIDER THE COMPLEXITIES ABOUT INSERTION (COMP-I), QUERY (COMP-Q), DELETION (COMP-D) AND MEMORY ACCESS (COMP-M).Parallel BF[94] supports multiattribute representation on network services. Besides, for online systems where elements may join and leave dynamically, several variants are proposed to realize capacity elasticity of BF, such as Dynamic BF[89], DBA[90], Par-BF[91], Scalable BF[118] and A 2 Buffering…”

A Bit String Content Aware Chunking Strategy for Reduced CPU Energy on Cloud Storage

“…Most existing chunking strategies [15][16][17] had used Rabin algorithm in signature calculation in which the total amount of chunk and the total number of signature calculations were nearly 3 orders of magnitude. There still existed a lot of invalid signature calculations.…”