Bloom Filter with a False Positive Free Zone

Kiss, Sándor Z.; Hosszú, Éva; Tapolcai, János; Rónyai, Lajos; Rottenstreich, Ori

doi:10.1109/infocom.2018.8486415

Cited by 30 publications

(12 citation statements)

References 46 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Top flow keys: When reporting the identities of the most influential flows, we cannot leverage data structures such as heap or priority queue to store the flow identities in the data plane. Instead, we leverage a packet mirroring feature in the switch with a Bloom filter [30,31,32]. Once the estimated statistic for a flow exceeds some threshold, the switch duplicates this packet and reports the copy to the controller.…”

Section: Methodsmentioning

confidence: 99%

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Liu¹,

Zhou²,

Rottenstreich³

et al. 2020

Symposium on Algorithmic Principles of Computer Systems

Self Cite

View full text Add to dashboard Cite

Network performance problems are notoriously difficult to diagnose. Prior profiling systems collect performance statistics by keeping information about each network flow, but maintaining per-flow state is not scalable on resource-constrained NIC and switch hardware. Instead, we propose sketch-based performance monitoring using memory that is sublinear in the number of flows. Existing sketches estimate flow monitoring metrics based on flow sizes. In contrast, performance monitoring typically requires combining information across pairs of packets, such as matching a data packet with its acknowledgment to compute a round-trip time. We define a new class of lean algorithms that use memory sublinear in both the size of input data and the number of flows. We then introduce lean algorithms for a set of important statistics, such as identifying flows with high latency, loss, out-of-order, or retransmitted packets. We implement prototypes of our lean algorithms on a commodity programmable switch using the P4 language. Our experiments show that lean algorithms detect ∼82% of top 100 problematic flows among real-world packet traces using just 40KB memory.

show abstract

Section: Methodsmentioning

confidence: 99%

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Liu¹,

Zhou²,

Rottenstreich³

et al. 2020

Symposium on Algorithmic Principles of Computer Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Most recently, Kiss et al propose EGH filter to replace the k hash functions {h 1 , · · · , h k } with the k simple functions {ĥ 1 , · · · ,ĥ k } generated based on k prime numbers. Intuitively, EGH filter supports the Bloom filter operations and additionally guarantees false positive free operations for a finite universe when a restricted number of elements stored in the filter [80]. In other words, given a finite universe set U with |U| elements, an EGH filter vector with m bits will not suffer from any false positive errors if at most n t elements are stored.…”

Section: Reducing Fp With Selected Hash Functionsmentioning

confidence: 99%

“…EABF [110] L-CBF [111] MLBF [142] Shifting BF [115] ICBF [20] Bloomier filter [95] Parallel BF [94] IBF [145] One hash BF [86] Ultra-fast BF [98] Double Buffering [137] A 2 Buffering [138] Forgetful BF [139] Performance Generalization EGH filter [80] Loglog BF [113] kBF [19] Fig. 13.…”

Section: Bit Vectormentioning

confidence: 99%

“…The computation complexity of these locality-sensitive hash functions is also a barrier of real implementation. Note that, the EGH filter [80], on the contrary, replaces the hash functions with some simple functions derived by prime numbers. By designing the parameters carefully, EGH filter guarantees a false positive free zone for a subset of elements.…”

Section: B Techniques Towards Hash Functionsmentioning

confidence: 99%

“…L-CBF [111] speeds up the query of CBF from the implementation perspective. EGH filter [80] also extends the bits as counters to enable the deletion functionality.…”

Section: Techniques Towards the Bit Vectormentioning

confidence: 99%

See 2 more Smart Citations

Optimizing Bloom Filter: Challenges, Solutions, and Comparisons

Luo

Guo

Richard

et al. 2019

IEEE Commun. Surv. Tutorials

Self Cite

173

View full text Add to dashboard Cite

Bloom filter (BF) has been widely used to support membership query, i.e., to judge whether a given element x is a member of a given set S or not. Recent years have seen a flourish design explosion of BF due to its characteristic of space-efficiency and the functionality of constant-time membership query. The existing reviews or surveys mainly focus on the applications of BF, but fall short in covering the current trends, thereby lacking intrinsic understanding of their design philosophy. To this end, this survey provides an overview of BF and its variants, with an emphasis on the optimization techniques. Basically, we survey the existing variants from two dimensions, i.e., performance and generalization. To improve the performance, dozens of variants devote themselves to reducing the false positives and implementation costs. Besides, tens of variants generalize the BF framework in more scenarios by diversifying the input sets and enriching the output functionalities. To summarize the existing efforts, we conduct an in-depth study of the existing literature on BF optimization, covering more than 60 variants. We unearth the design philosophy of these variants and elaborate how the employed optimization techniques improve BF. Furthermore, comprehensive analysis and qualitative comparison are conducted from the perspectives of BF components. Lastly, we highlight the future trends of designing BFs. This is, to the best of our knowledge, the first survey that accomplishes such goals.

show abstract

Query the trajectory based on the precise track: a Bloom filter-based approach

et al. 2021

View full text Add to dashboard Cite

Fast and precise querying in a given set of trajectory points is an important issue of trajectory query. Typically, there are massive trajectory data in the database, yet the query sets only have a few points, which is a challenge for the superior performance of trajectory querying. The current trajectory query methods commonly use the tree-based index structure and the signature-based method to classify, simplify, and filter the trajectory to improve the performance. However, the unstructured essence and the spatiotemporal heterogeneity of the trajectory-sequence lead these methods to a high degree of spatial overlap, frequent I/O, and high memory occupation. Thus, they are not suitable for the time-critical tasks of trajectory big data. In this paper, a query method of trajectory is developed on the Bloom Filter. Based on the gridded space and geocoding, the spatial trajectory sequences (tracks) query is transformed into the query of the text string. The geospace was regularly divided by the geographic grid, and each cell was assigned an independent geocode, converting the high-dimensional irregular space trajectory query into a one-dimensional string query. The point in each cell is regarded as a signature, which forms a mapping to the bit-array of the Bloom Filter. This conversion effectively eliminates the high degree of overlap and instability of query performance. Meanwhile, the independent coding ensures the uniqueness of the whole tracks. In this method, there is no need for additional I/O on the raw trajectory data when the track is queried. Compared to the original data, the memory occupied by this method is negligible. Based on Beijing Taxi and Shenzhen bus trajectory data, an experiment using this method was constructed, and random queries under a variety of conditions boundaries were constructed. The results verified that the performance and stability of our method, compared to R*tree index, have been improved by 2000 to 4000 times, based on one million to tens of millions of trajectory data. And the Bloom Filter-based query method is hardly affected by grid size, original data size, and length of tracks. With such a time advantage, our method is suitable for time-critical spatial computation tasks, such as anti-terrorism, public safety, epidemic prevention, and control, etc.

show abstract

Bloom Filter with a False Positive Free Zone

Cited by 30 publications

References 46 publications

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Optimizing Bloom Filter: Challenges, Solutions, and Comparisons

Query the trajectory based on the precise track: a Bloom filter-based approach

Contact Info

Product

Resources

About