Detection of Superpoints Using a Vector Bloom Filter

Super point is a special host in network which communicates with lots of other hosts in a certain time period. The number of hosts contacting with a super point is called as its cardinality. Cardinality estimating plays important roles in network management and security. All of existing works focus on how to estimate super point's cardinality under discrete time window. But discrete time window causes great delay and the accuracy of estimating result is subject to the starting of the window. sliding time window, moving forwarding a small slice every time, offers a more accuracy and timely scale to monitor super point's cardinality. On the other hand, super point's cardinality estimating under sliding time window is more difficult because it requires an algorithm can record the cardinality incrementally and report them immediately at the end of the sliding duration. This paper firstly solves this problem by devising a sliding time window available algorithm SRLA. SRLA consists of two cardinality estimating algorithms, sliding rough estimator SRE and sliding linear estimator SLE. SRE is used to detect super point while scanning packets and it generates a candidate super point list at the end of a sliding time window. With this candidate super point list, SLE estimates the cardinality of every candidate super point fast and accurately. SRLA's ability of working under sliding time window comes from a novel cardinality recorder, distance recorder DR. DR records the time a host appearing and helps SRE and SLE to judge if a host contacting with a super point is in a certain time period. SRLA could run parallel to deal with high speed network in line speed. This paper also gives the way to deploy SRLA on a common GPU. Experiments on real world traffics which have 40 GB/s bandwidth show that SRLA estimates super point's cardinality within 100 milliseconds under sliding time window when running on a Nvidia GPU GTX650 with 1 GB memory. The estimating time of SRLA is much smaller than that of other algorithms which consumes more than 2000 milliseconds under discrete time window.

show abstract

“…Liu et al [15] proposed a simple method to restore super hosts basing on bloom filter. They called this algorithm as Vector Bloom Filter(VBF).…”

Section: Related Workmentioning

confidence: 99%

SRLA: A Real Time Sliding Time Window Super Point Cardinality Estimation Algorithm for High Speed Network Based on GPU

Ding

Gong

et al. 2018

2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference On

Self Cite

View full text Add to dashboard Cite

show abstract

“…Liu et al [13] proposed a simple method to restore super hosts basing on bloom filter. They called this algorithm as Vector Bloom Filter(VBF).…”

Section: Related Workmentioning

confidence: 99%

“…There are total 1 GB of graphic memory in this GPU card and it communicates with computer through PCIe3.0 which has a bandwidth as high as 16 Gb/s. We compare the performance of different algorithms: DCDS [23], VBFA [13], GSE [18] and SLGA. SLGA is the one proposed in this paper.…”

Section: Experiments and Analysismentioning

confidence: 99%

“…A high accuracy algorithm should detect out all super points and does not report normal hosts as super points by mistake. Many researchers try to use small and fast memory, such as static random accessing memory SRAM [13] [14], to detect super point. These algorithms used estimating method to record hosts' cardinalities, the opposite hosts number during a time period.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Most Memory Efficient Distributed Super Points Detection on Core Networks

Ding

2018

Algorithms and Architectures for Parallel Processing

View full text Add to dashboard Cite

The super point, a host which communicates with lots of others, is a kind of special hosts gotten great focus. Mining super point at the edge of a network is the foundation of many network research fields. In this paper, we proposed the most memory efficient super points detection scheme. This scheme contains a super points reconstruction algorithm called short estimator and a super points filter algorithm called long estimator. Short estimator gives a super points candidate list using thousands of bytes memory and long estimator improves the accuracy of detection result using millions of bytes memory. Combining short estimator and long estimator, our scheme acquires the highest accuracy using the smallest memory than other algorithms. There is no data conflict and floating operation in our scheme. This ensures that our scheme is suitable for parallel running and we deploy our scheme on a common GPU to accelerate processing speed. We also describe how to extend our algorithm to sliding time. Experiments on several real-world core network traffics show that our algorithm acquires the highest accuracy with only consuming littler than one-fifth memory of other algorithms.

show abstract

“…Since proposed by Bloom Burton in 1970 [15], it has been used in database applications, and some improved algorithms are proposed [16][17]. In recent years, it has been widely concerned in the field of network [18][19].…”

Section: Bloom Filter and Multistage Bloom Filtersmentioning

confidence: 99%