Constant Time Updates in Hierarchical Heavy Hitters

Basat, Ran Ben; Einziger, Gil; Friedman, Roy; Luizelli, Marcelo Caggiani; Waisbard, Erez

doi:10.1145/3098822.3098832

Cited by 117 publications

(72 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Recent work [41] also leverage switches to efficiently point to distributed end-host information for whole network visibility. Switch-based monitoring tools: Since hardware switches have limited memory resources for monitoring tasks, memoryoptimized sketching algorithms have been proposed to a variety of flow monitoring tasks, such as heavy hitters (frequent flows) [42,14,43,15], detecting hierarchical heavy hitters [20,21], counting distinct flows [44,18], estimating frequency moments [11], and change detection [45,18]. These sketching algorithms offer worst-case guarantees to arbitrary network workloads and use sublinear memory in terms of the number of distinct network flows.…”

Section: Related Workmentioning

confidence: 99%

“…Our design is inspired by the rich literature of sketching algorithms in network flow monitoring, where the traffic is modeled as a stream of elements [11]. A number of sketching algorithms have been introduced to accurately estimate various flow metrics such as heavy hitters [12,13,14,15,16,17,18], hierarchical heavy hitters [19,20,21], flow size distribution [12,22,23], and change detection [17,18]. These algorithms allow for memory-efficient measurement systems while maintaining guaranteed fidelity.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Liu¹,

Zhou²,

Rottenstreich³

et al. 2020

Symposium on Algorithmic Principles of Computer Systems

View full text Add to dashboard Cite

Network performance problems are notoriously difficult to diagnose. Prior profiling systems collect performance statistics by keeping information about each network flow, but maintaining per-flow state is not scalable on resource-constrained NIC and switch hardware. Instead, we propose sketch-based performance monitoring using memory that is sublinear in the number of flows. Existing sketches estimate flow monitoring metrics based on flow sizes. In contrast, performance monitoring typically requires combining information across pairs of packets, such as matching a data packet with its acknowledgment to compute a round-trip time. We define a new class of lean algorithms that use memory sublinear in both the size of input data and the number of flows. We then introduce lean algorithms for a set of important statistics, such as identifying flows with high latency, loss, out-of-order, or retransmitted packets. We implement prototypes of our lean algorithms on a commodity programmable switch using the P4 language. Our experiments show that lean algorithms detect ∼82% of top 100 problematic flows among real-world packet traces using just 40KB memory.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Liu¹,

Zhou²,

Rottenstreich³

et al. 2020

Symposium on Algorithmic Principles of Computer Systems

View full text Add to dashboard Cite

show abstract

“…CM [17], 4 CU [20], Count [21], and Elastic. For CM, CU, and Count, we use 3 hash functions as recommended in [62].…”

Section: A Experimental Setupmentioning

confidence: 99%

Adaptive Measurements Using One Elastic Sketch

Yang

Jiang

Liu

et al. 2019

IEEE/ACM Trans. Networking

View full text Add to dashboard Cite

When network is undergoing problems such as congestion, scan attack, DDoS attack, etc, measurements are much more important than usual. In this case, traffic characteristics including available bandwidth, packet rate, and flow size distribution vary drastically, significantly degrading the performance of measurements. To address this issue, we propose the Elastic sketch. It is adaptive to currently traffic characteristics. Besides, it is generic to measurement tasks and platforms. We implement the Elastic sketch on six platforms: P4, FPGA, GPU, CPU, multi-core CPU, and OVS, to process six typical measurement tasks. Experimental results and theoretical analysis show that the Elastic sketch can adapt well to traffic characteristics. Compared to the state-of-the-art, the Elastic sketch achieves 44.6 ∼ 45.2 times faster speed and 2.0 ∼ 273.7 smaller error rate.

show abstract

“…Finally, we apply (6) to conclude that once x arrives with a cumulative volume of t · M · , it will never be evicted (Line 7) and from that moment on its volume will be measured exactly.…”

Section: Proof Of Lemmamentioning

confidence: 99%

Fast Flow Volume Estimation

Basat

Einziger

Friedman

2018

Proceedings of the 19th International Conference on Distributed Computing and Networking

Self Cite

View full text Add to dashboard Cite

The increasing popularity of jumbo frames means growing variance in the size of packets transmitted in modern networks. Consequently, network monitoring tools must maintain explicit traffic volume statistics rather than settle for packet counting as before. We present constant time algorithms for volume estimations in streams and sliding windows, which are faster than previous work. Our solutions are formally analyzed and are extensively evaluated over multiple real-world packet traces as well as synthetic ones. For streams, we demonstrate a run-time improvement of up to 2.4X compared to the state of the art. On sliding windows, we exhibit a memory reduction of over 100X on all traces and an asymptotic runtime improvement to a constant. Finally, we apply our approach to hierarchical heavy hitters and achieve an empirical 2.4-7X speedup.

show abstract

Constant Time Updates in Hierarchical Heavy Hitters

Cited by 117 publications

References 41 publications

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Adaptive Measurements Using One Elastic Sketch

Fast Flow Volume Estimation

Contact Info

Product

Resources

About