An Efficient <i>K</i>-Persistent Spread Estimator for Traffic Measurement in High-Speed Networks

Huang, He; Sun, Yu-E; Ma, Chaoyi; Chen, Shigang; Zhou, You; Yang, Wenjian; Tang, Shaojie; Xu, Hongli; Qiao, Yan

doi:10.1109/tnet.2020.2982003

Cited by 31 publications

(8 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides, some variants of supernode detection are proposed in the literature [52,53]. Zhou et al [52] proposed the solution of persistent spread problem, which counts the number of distinct elements in each flow persistently occurring in the predefined measurement periods.…”

Section: Related Workmentioning

confidence: 99%

An Efficient Method for Detecting Supernodes Using Reversible Summary Data Structures in the Distributed Monitoring Systems

Zhou

Jin

2022

Security and Communication Networks

View full text Add to dashboard Cite

Supernode detection has many applications in detecting network attacks, assisting resource allocation, etc. As 5G/IoT networks constantly grow, big network traffic brings a great challenge to collect massive traffic data in compact and real-time way. Previous works focus on detecting supernodes in a measurement point, while only a few works consider it in the distributed monitoring system. Moreover, they are not able to measure two types of node cardinalities simultaneously and reconstruct labels of supernodes efficiently due to large calculation and memory cost. To address these problems, we propose a novel reversible and distributed traffic summarization called RDS to simultaneously measure source and destination cardinalities for detecting supernodes in the distributed monitoring system. The basic idea of our approach is that each monitor generates a summary data structure using the coming packets and sends the summary data structure to the controller; then, the controller aggregates the received summary data structures, estimates node cardinalities, and reconstructs labels of supernodes according to the aggregated summary data structure. The experimental results based on real network traffic demonstrate that the proposed approach can detect up to 96% supernodes with a low memory requirement in comparison with state-of-the-art approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

An Efficient Method for Detecting Supernodes Using Reversible Summary Data Structures in the Distributed Monitoring Systems

Zhou

Jin

2022

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…Our focus in this paper is on estimation of flow spread. Past solutions use hashbased compact data structures called sketches [11], [12], [13], [18], [14], [15], [19], [20], [21], [3], [22], [23], [24]. These solutions can be divided into three categories.…”

Section: Other Related Workmentioning

confidence: 99%

“…As a result, the total memory allocation is proportional to the number of flows being monitored. The second category estimates the spreads of multiple flows by using a shared pool of resources, either bits or counters [14], [15], [19], [20], [21]. This is done by constructing virtual sketches from the shared memory.…”

Section: Other Related Workmentioning

confidence: 99%

Hierarchical Virtual Bitmaps for Spread Estimation in Traffic Measurement

Odegbile¹,

Ma²,

Chen³

et al. 2021

Computer Science &Amp; Information Technology (CS &Amp; IT)

Self Cite

View full text Add to dashboard Cite

This paper introduces a hierarchical traffic model for spread measurement of network traffic flows. The hierarchical model, which aggregates lower level flows into higher-level flows in a hierarchical structure, will allow us to measure network traffic at different granularities at once to support diverse traffic analysis from a grand view to fine-grained details. The spread of a flow is the number of distinct elements (under measurement) in the flow, where the flow label (that identifies packets belonging to the flow) and the elements (which are defined based on application need) can be found in packet headers or payload. Traditional flow spread estimators are designed without hierarchical traffic modeling in mind, and incur high overhead when they are applied to each level of the traffic hierarchy. In this paper, we propose a new Hierarchical Virtual bitmap Estimator (HVE) that performs simultaneous multi-level traffic measurement, at the same cost of a traditional estimator, without degrading measurement accuracy. We implement the proposed solution and perform experiments based on real traffic traces. The experimental results demonstrate that HVE improves measurement throughput by 43% to 155%, thanks to the reduction of perpacket processing overhead. For small to medium flows, its measurement accuracy is largely similar to traditional estimators that work at one level at a time. For large aggregate and base flows, its accuracy is better, with up to 97% smaller error in our experiments.

show abstract

“…It obtains far better memory efficiency. Huang et al [23] proposed an efficient and accurate k-persistent estimator. It uses SUM to join the information collected from different periods to estimate the k-persistent spread of each flow.…”

Section: Related Workmentioning

confidence: 99%

“…If the estimation 􏽢 f j c is less than F c , the corresponding IDs are not considered to be users with persistent behavior. Otherwise, we reconstruct IDs of users with persistent behavior by (23). e estimation of their occurrence frequency is expressed as Input: the updated B i , 1 ≤ i ≤ r Output: the estimated occurrence frequency (1) for each user u do (2) for each timeslot t do (3) compute…”

Section: Mathematical Problems In Engineeringmentioning

confidence: 99%

Detecting Persistent User Behavior Using Probabilistic Counting in Network-Wide View

Zhou

Jin

2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Persistent user behavior monitoring, which deals with finding users that occur persistently over a measurement period, is one hot topic in traffic measurement. It is significant for many applications, such as anomaly detection. Former works concentrate on monitoring frequent user behavior, such as users occurring frequently either over one measurement period or on one monitor. They have paid little attention to detect persistent user behavior over a long measurement period on multiple monitors. However, persistent users do not necessarily appear frequently in a short measurement period, but appear persistently in a long measurement period. Due to limited resource on monitors, it is not practical to collect a tremendous amount of network traffic in a long measurement period on one single monitor. Moreover, since network attackers deliberately send packets flowing through the entire managed network, it is difficult to detect abnormal behavior on one single monitor. To solve the above challenges, a novel method for detecting persistent user behavior called DPU is proposed, and it contains both online distributed traffic collection in a long measurement period on multiple monitors and offline centralized user behavior detection on the central server. The key idea of DPU is that we design the compact distributed synopsis data structure to collect the relevant information with users occurring in a long measurement period on each monitor, and we can reconstruct user IDs using simple calculations and bit settings to find users with persistent behavior on the basis of estimated occurrence frequency of users on the central server when user IDs are unknown in advance. The experiments are conducted on real traffic to evaluate the performance of detecting persistent user behavior, and the experimental results illustrate that our method can improve about 30% estimation accuracy, 40% detection precision, and accelerate about 3 times in comparison with the related method.

show abstract

An Efficient K-Persistent Spread Estimator for Traffic Measurement in High-Speed Networks

Cited by 31 publications

References 31 publications

An Efficient Method for Detecting Supernodes Using Reversible Summary Data Structures in the Distributed Monitoring Systems

An Efficient Method for Detecting Supernodes Using Reversible Summary Data Structures in the Distributed Monitoring Systems

Hierarchical Virtual Bitmaps for Spread Estimation in Traffic Measurement

Detecting Persistent User Behavior Using Probabilistic Counting in Network-Wide View

Contact Info

Product

Resources

About