Identifying Heavy-Hitter Flows from Sampled Flow Statistics

Abstract

SUMMARYWith the rapid increase of link speed in recent years, packet sampling has become a very attractive and scalable means in collecting flow statistics; however, it also makes inferring original flow characteristics much more difficult. In this paper, we develop techniques and schemes to identify flows with a very large number of packets (also known as heavy-hitter flows) from sampled flow statistics. Our approach follows a two-stage strategy: We first parametrically estimate the original flow length distribution from sampled flows. We then identify heavy-hitter flows with Bayes' theorem, where the flow length distribution estimated at the first stage is used as an a priori distribution. Our approach is validated and evaluated with publicly available packet traces. We show that our approach provides a very flexible framework in striking an appropriate balance between false positives and false negatives when sampling frequency is given.

Characterization of IP flows eligible for lambda-connections in optical networks

“…The characterization of big IP flows in large scale optical networks has also been addressed in other research works [13] [14] [15] [16] [12]. From this related work, we highlight the following two investigations, due to some similarities to this paper.…”

“…The knowledge of the heavy-hitter 1 [12] of flows originated from these applications allows network managers to establish lambda-connections in advance for such flows. However, there may be also other big IP flows in current networks that could also benefit from being moved to lambda-connections, but for some reason (e.g., the network manager may not be aware of their existence) they may not be selected.…”

Human error tolerant anomaly detection based on time-periodic packet sampling

“…Although packet sampling provides greater scalability for network measurements [11,[15][16][17][18], it makes inferring the original traffic characteristics much more difficult and biased because it is inherently lossy. For example, Kawahara et al [19] showed that network anomalies generating a large number of small flows, such as network scans or SYN flooding, become difficult to detect during packet sampling.…”

Reliability evaluation according to a routing scheme for multi-state computer networks under assured accuracy rate

“…Besides, numerous researchers have discussed the issues related to considered flow accuracy when measuring local area network traffic (Amer 1982;Chlamtac 1980;Jain and Routhier 1986). Because high packet-rate flows have a great impact on network performance, their prompt identification is important in network management and traffic engineering (Mori et al 2007).…”