Identifying frequent flows in large datasets through probabilistic bloom filters

Yao, Yanjun; Xiong, Sisi; Liao, Jilong; Berry, Michael W.; Qi, Hairong; Cao, Qing

doi:10.1109/iwqos.2015.7404747

Cited by 6 publications

(6 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As discussed in our previous paper [7], the estimation accuracy of the PBF and the T-PBF would be poor if an improper combination of parameters is chosen and used. In the following, we demonstrate that our algorithms for parameter selection will 280 always lead to Nash equilibriums in practice, i.e., further adjusting any parameter by itself will not lead to a better performance [36].…”

Section: Pbf Parameter Selections Are Nash Equilibriummentioning

confidence: 99%

“…Next, we calculate the confidence interval for f , by approximating it using a normal distribution based on the central limit theorem. This is also the so-called Wald Method, and we can derive the lower and upper bounds for f as shown below [7]:…”

Section: Performance Modeling Of Pbfsmentioning

confidence: 99%

“…To perform accurate frequency estimations with T-PBF, it is critical to choose proper values for parameters, such as m, k, p, and q. As T-PBF extends PBF by introducing one additional parameter q, and the decaying operations are only triggered 255 at the end of each period, we can use the same algorithms to choose the values of m, k, and p, based on what we discussed in our previous paper [7]. As a result, in the following part, we will discuss how to choose the value for q.…”

Section: Selection Of Parameters For T-pbfmentioning

confidence: 99%

“…In our previous work [7], we developed the probabilistic Bloom filter (PBF), which extends a highly memory compact probabilistic data structure, the classical Bloom filter, to support frequency queries. The key difference is that whenever a flow is inserted, instead of flipping the hash locations from 0 to 1 deterministically, we introduce a new 25 parameter p, ranging from 0 to 1, which represents the probability to flip the bit.…”

Section: Introductionmentioning

confidence: 99%

“…According to analysis in our previous paper [7], through carefully parameter tuning, the PBF can also configured as a per-flow frequency estimation. However, since the PBF is a bit-array and it does not store flow identifiers explicitly, we also use a Initialize P BF 5: for i = 1 → n do 6: f id = f low id(pi) 7:…”

mentioning

confidence: 99%

See 4 more Smart Citations

Frequent traffic flow identification through probabilistic bloom filter and its GPU-based acceleration

Xiong

Yao

Berry

et al. 2017

Journal of Network and Computer Applications

Self Cite

View full text Add to dashboard Cite

Traffic measurement and monitoring are critical for many network applications, for example, bandwidth management and detecting security threats such as DoS (Denial of Service) attacks. In these cases, traffic is usually modeled as a collection of flows. One central problem is to identify those frequent flows, which account for a large percentage of total traffic, or whose frequency exceeds the user-specified threshold. In this paper, we describe a novel data structure called the probabilistic Bloom filter (PBF), which extends the classical Bloom filter into probabilistic direction, so that it can effectively estimate flows' frequencies, and identify frequent flows. We analyze the performance, tradeoffs, and capacity of this data structure, and investigate how to calibrate this data structure's parameters. We further develop one extension of the PBF for dynamic data stream needs. We implement our PBF on GPUs to gain more time efficiency. By testing with real network traces collected from a backbone router, we demonstrate that our method can keep track of flows' frequencies with adjustable accuracy, so that frequent flows can be identified with constant computational time complexity and low memory overhead.However, our previous work left several problems open. First, there is no rigorous analysis on how to select the PBF's parameters, where a poorly selected set of parameters will cause PBF to generate inaccurate estimates. Second, the PBF was developed for handling static data, but real-world traffic datasets may usually exhibit streaming

show abstract

Section: Pbf Parameter Selections Are Nash Equilibriummentioning

confidence: 99%

Section: Performance Modeling Of Pbfsmentioning

confidence: 99%

Section: Selection Of Parameters For T-pbfmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Frequent traffic flow identification through probabilistic bloom filter and its GPU-based acceleration

Xiong

Yao

Berry

et al. 2017

Journal of Network and Computer Applications

Self Cite

View full text Add to dashboard Cite

show abstract

Hunting the pertinency of hash and bloom filter combinations on GPU for fast pattern matching

Bhat

Thilak

Vaibhav

2022

Int. j. inf. tecnol.

View full text Add to dashboard Cite

There has been rapid growth in the field of graphical processing unit (GPU) programming due to the drastic increase in the computing hardware manufacturing. The technology used in these devices is now more affordable and accessible to the general public. With this growth, many serial programming applications that are now being transformed into more efficient parallel programming applications with significant improvement in the performance. The best example for this is parallel implementation of the probabilistic data structure Bloom filter in set membership queries. However, despite of it’s remarkable performance in speed and memory usage, there is a computational overhead in the calculation of hashes in Bloom filter. In this paper, the impact of the choice of hash functions on the qualitative properties of the Bloom filter has been experimentally recorded and the results show that there is a possibility of large performance gap among various hash functions. We have implemented the Bloom filter based pattern matching technique on GPU using compute unified device architecture (CUDA) and benchmark the performance of several cryptographic and non-cryptographic hash functions.

show abstract

Identification of heavy hitters for network data streams with probabilistic sketch

Zhou

Zhu

Liu

et al. 2018

2018 IEEE 3rd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA)

View full text Add to dashboard Cite

Identifying frequent flows in large datasets through probabilistic bloom filters

Cited by 6 publications

References 21 publications

Frequent traffic flow identification through probabilistic bloom filter and its GPU-based acceleration

Frequent traffic flow identification through probabilistic bloom filter and its GPU-based acceleration

Hunting the pertinency of hash and bloom filter combinations on GPU for fast pattern matching

Identification of heavy hitters for network data streams with probabilistic sketch

Contact Info

Product

Resources

About