You Can Drop but You Can't Hide: &lt;tex&gt;$K$&lt;/tex&gt;-persistent Spread Estimation in High-speed Networks

Huang, He; Sun, Yu-E; Chen, Shigang; Tang, Shaojie; Han, Kai; Yuan, Jing; Yang, Wenjian

doi:10.1109/infocom.2018.8485998

Cited by 35 publications

(7 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Filter [58] Virtual Register Sharing [67] Virtual HyperLogLog [70] Counter tree [75] Braids [22] SketchVisor [135] MAPLE [160] NitroSketch [55], R HHH [54] CASE [174] CASE [183], RCC [87] HeavyGuardian [100] Sliding HLL [127] Fast sketch [120], OPA [115] Sequential hashing [112] Modular hashing [119] Bitcount [143], CBF [50] SketchLearn [20] Group Testing [159] HeavyKeeper [99] HeavyGuardian [100] ICE-Buckets [52] HyperSight [153] MultiResBitmap [170] HLL-TailCut+ [74], RAP [93], Frequent [94] ACE [125], Conservative update [88] CountMax [96], LCF [184], LR(T) [185], SpaceSaving [95], CSketch [89], CSM [90] CHHFR [171], Refined LL [166] CounterMap [198], IMP [118] SketchVisor [135], WCSS[128] SWAMP [129], RCD [113] Defeat …”

Section: Improve Accuracymentioning

confidence: 99%

“…k out of t persistence. Although we can separate attackers from the legitimates by finding flows appearing during all measurement periods, Huang et al [160] considered the situation that attackers might give up sending packets during several periods. To solve this problem, they formulate a new problem called k-persistent spread estimation, which measures persistent traffic elements in each flow that appear during at least k out of t measurement periods.…”

Section: ) Time Dimensionmentioning

confidence: 99%

See 1 more Smart Citation

A survey of sketches in traffic measurement: Design, Optimization, Application and Implementation

Li¹,

Luo²,

Guo³

et al. 2020

Preprint

View full text Add to dashboard Cite

Network measurement probes the underlying network to support upper-level decisions such as network management, network update, network maintenance, network defense and beyond. Due to the massive, speedy, unpredictable features of network flows, sketches are widely implemented in measurement nodes to record the frequency or estimate the cardinality of flows approximately. At their cores, sketches usually maintain one or multiple counter array(s), and relies on hash functions to select the counter(s) for each flow. Then the space-efficient sketches from the distributed measurement nodes are aggregated to provide statistics of the undergoing flows. Currently, tremendous redesigns and optimizations have been proposed to further improve the sketches for better network measurement performance. However, the existing reviews or surveys mainly focus on one particular aspect of measurement tasks. Researchers and engineers in the network measurement community desire an all-in-one survey which covers the whole processing pipeline of sketch-based network measurement. To this end, we present the first comprehensive survey in this area. We first introduce the preparation of flows for measurement, then detail the most recent investigations of design, aggregation, decoding, application and implementation of sketches for network measurement. To summary the existing efforts, we conduct an in-depth study of the existing literature, covering more than 80 sketch designs and optimization strategies. Furthermore, we conduct a comprehensive analysis and qualitative/quantitative comparison of the sketch designs. Finally, we highlight the open issues for future sketch-based network measurement research.

show abstract

Section: Improve Accuracymentioning

confidence: 99%

Section: ) Time Dimensionmentioning

confidence: 99%

A survey of sketches in traffic measurement: Design, Optimization, Application and Implementation

Li¹,

Luo²,

Guo³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Network traffic measurement is another branch of the traffic measurement, which is similar to transportation traffic measurement. Various methods [2], [3], [8], [13], [28], [29] have been proposed for network traffic measurement, which is to measure the network traffic in a network router. However, we need to protect the trajectory privacy of vehicles in transportation traffic measurement, which is not considered in the network traffic measurement.…”

Section: Related Workmentioning

confidence: 99%

Persistent Traffic Measurement through Vehicle-to-Infrastructure Communications in Cyber-Physical Road Systems

Sun

Huang

Chen

et al. 2019

IEEE Trans. on Mobile Comput.

Self Cite

View full text Add to dashboard Cite

Measuring traffic volume in a road system has important applications in transportation engineering. The connected vehicle technologies integrate wireless communications and computers into transportation systems, allowing wireless data exchanges between vehicles and road-side equipment, and enabling large-scale, sophisticated traffic measurement. This paper investigates the problem of persistent traffic measurement, which was not adequately studied in the prior art, particularly in the context of intelligent vehicular networks. We propose three estimators for privacy-preserving persistent traffic measurement: one for point traffic, one for point-to-point traffic, and another for three-point traffic. After that, we present a general framework to measure persistent traffic that go through more than three locations. The estimators are mathematically derived from the join result of traffic records, which are produced by the electronic roadside units with privacy-preserving data structures. We evaluate our estimation methods using simulations based on both real transportation traffic data and synthetic data. The numerical results demonstrate the effectiveness of the proposed methods in producing high measurement accuracy and allowing accuracy-privacy tradeoff through parameter setting.

show abstract

“…A CCURATE and timely network traffic measurement is an essential part of network security management and network information forensics, e.g., host cardinality measurement [1]- [5], flow size measurement [6]- [10], abnormal behavior measurement [11]- [15], and persistent spread measurement [16], [17]. Among these applications, host cardinality measurement is a distinctive task.…”

Section: Introductionmentioning

confidence: 99%

SuperSketch: A Multi-Dimensional Reversible Data Structure for Super Host Identification

Jing

Han

Yan

et al. 2022

IEEE Trans. Dependable and Secure Comput.

View full text Add to dashboard Cite

Facing big network traffic data, effective data compression becomes crucially important and urgently needed for estimating host cardinalities and identifying super hosts. However, the current literature confronts several challenges: incapability of simultaneously measuring various types of host cardinalities and inability to efficiently reconstruct super host addresses. To address these challenges, in this paper, we propose a novel sketch data structure, named SuperSketch, to simultaneously measure multiple types of host cardinalities with the purpose of efficiently identifying super hosts. SuperSketch has two significant characteristics: multi-dimensionality and reversibility. The multi-dimensionality makes SuperSketch capable of simultaneously measuring Source Cardinality, Destination Cardinality and Destination Port Cardinality. The reversibility allows SuperSketch to accurately and quickly reconstruct the original addresses of super hosts once they are identified. We conduct both theoretical analysis and performance evaluation based on real-world network traffic. Experimental results show that SuperSketch achieves outstanding performance for multi-cardinality measurement, super host identification and host address reconstruction.

show abstract

You Can Drop but You Can't Hide: <tex>$K$</tex>-persistent Spread Estimation in High-speed Networks

Cited by 35 publications

References 17 publications

A survey of sketches in traffic measurement: Design, Optimization, Application and Implementation

A survey of sketches in traffic measurement: Design, Optimization, Application and Implementation

Persistent Traffic Measurement through Vehicle-to-Infrastructure Communications in Cyber-Physical Road Systems

SuperSketch: A Multi-Dimensional Reversible Data Structure for Super Host Identification

Contact Info

Product

Resources

About