A Flexible Sketch-Based Network Traffic Monitoring Infrastructure

Abstract

Sketch-based data streaming algorithms are used in many network traffic monitoring applications to obtain accurate estimates of traffic flow statistics. However, the current implementation of sketch-based methods in network monitoring is too application-specific. Their flexibility is limited as the hardware implementation of sketch data structure in the network device depends on the measurement tasks. The sketch counters only summarize flow statistics that are relevant to a specific measurement task and cannot be reused for different measurement tasks. In this paper, we propose FlexSketchMon, a system designed to provide the flexibility of using various sketch-based algorithms for traffic monitoring and measurement tasks. FlexSketchMon leverages on a novel data plane architecture that collects traffic flow statistics and provides arbitrary flow aggregations to the monitoring applications. The data plane design comprises a flow counter table and a flow key table for storing flow-level data. FlexSketchMon is implemented on the NetFPGA-SUME platform and is capable of processing network traffic at line rate in a worst-case scenario corresponding to a 64-byte minimum Ethernet frame size. The update of the flow counter table, which is the critical path in the proposed system, can achieve a throughput of 96 Gbps. The simulation results based on a real-world network traffic traces for three monitoring applications-estimation, superspreader detection, and heavy hitter detection-are presented to demonstrate the performance of FlexSketchMon. The results show that FlexSketchMon yields comparable and better measurement accuracy compared to previous approaches.

Exploring Network-Wide Flow Data With Flowyager

“…the supported queries. Data summaries-Sketches: Another approach for computing network summaries are sketches, e.g., [73], [34], [74] as well as systems that utilize sketches for network monitoring and debugging [75], [25], [35], [76], [77]. The capabilities of sketches include counting, top-K, HH, as well as HHH.…”

SuperSketch: A Multi-Dimensional Reversible Data Structure for Super Host Identification

“…Existing sketches used to measure host cardinalities are in a form of two-dimension [1]- [3], [32], [34], which can only monitor a single type of host cardinality. Applying application-specific cardinality measurement methods are not flexible because we need to construct sketches with many different keys, which cannot satisfy the requirements of global network security measurement [35]. Moreover, the one-pass constraint of big-volume traffic data brings additional challenges to simultaneously measure various types of host cardinalities [22], [36].…”