Diagnosing network-wide traffic anomalies

Lakhina, Anukool; CrovellaMark,; DiotChristophe,

doi:10.1145/1030194.1015492

Cited by 395 publications

(232 citation statements)

References 28 publications

Supporting

Mentioning

191

Contrasting

Order By: Relevance

“…Note that several papers in the networking literature [14,25,26] have relied on a space-time analysis of network traffic, similarly to this paper. Those studies are similar to our work in that they use PCA or Kalman filtering in order to identify abnormal patterns in large traffic datasets.…”

Section: Traffic On the As Topologymentioning

confidence: 99%

“…GÉANT, towards all destinations to which the observation network sends traffic. References [14,25,26] focus on the paths of the traffic observed by some network, but only the part that crosses that network. Our work is the first to try to characterize the space-time dynamics of traffic as seen by a large network provider as it crosses the whole Internet topology.…”

Section: Traffic On the As Topologymentioning

confidence: 99%

“…Contrary to [14,25,26] who also take a time-space approach to analyze network traffic, we do not limit the network topology to a single network. Rather, we rely on the whole network-level path that crosses the studied network until it reaches the destination.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On the complexity of Internet traffic dynamics on its topology

Uhlig

2009

Telecommun Syst

View full text Add to dashboard Cite

Most studies of Internet traffic rely on observations from a single link. The corresponding traffic dynamics has been studied for more than a decade and is well understood. The study of how traffic on the Internet topology, on the other hand, is poorly understood and has been largely limited to the distribution of traffic among sourcedestination pairs inside the studied network, also called the traffic matrix. In this paper, we make a first step towards understanding the way traffic gets distributed onto the whole topology of the Internet. For this, we rely on the traffic seen by a transit network, for a period of more than a week. As we are still at the stage of understanding the topological traffic distribution, we do not try to model the traffic dynamics. Rather we concentrate on understanding the complexity of describing the traffic observed by a transit network, how it maps onto the AS-level topology of the Internet and how it changes over time. For this, we rely on well-known tools of multi-variate analysis and multi-resolution analysis.Our first observation is that the structure of the Internet topology highly impacts the traffic distribution. Second, our attempts at compressing the traffic on the topology through dimension reduction suggests two options for traffic modeling: (1) to ignore links on the topology for which we do not see much traffic, or (2) to ignore time scales smaller than a few hours. In either case, important properties of the traffic might be lost, so might not be an option to build realistic models of Internet traffic.Realistic models of Internet traffic on the topology are not out of reach though. In this paper, we identify two prop-S. Uhlig ( ) erties such models should have: (1) use a compact representation of the dependencies of the traffic on the topology, and (2) be able to capture the complex multi-scale nature of traffic dynamics on different types of links.

show abstract

Section: Traffic On the As Topologymentioning

confidence: 99%

Section: Traffic On the As Topologymentioning

confidence: 99%

See 1 more Smart Citation

On the complexity of Internet traffic dynamics on its topology

Uhlig

2009

Telecommun Syst

View full text Add to dashboard Cite

show abstract

“…Techniques such as Principal Component Analysis and Independent Component Analysis have been used to identify and retain independent variables and remove the redundant and derived metrics [14]. Data compression techniques have also been used to define the purging and retention policies [15].…”

Section: Related Workmentioning

confidence: 99%

Adaptive Monitoring: Application of Probing to Adapt Passive Monitoring

Jeswani

Natu²,

Ghosh

2014

J Netw Syst Manage

View full text Add to dashboard Cite

Availability of good quality monitoring data is a vital need for management of today's data centers. However, effective use of monitoring tools demands an understanding of the monitoring requirements that system administrators most often lack. Instead of a well-defined process of defining a monitoring strategy, system administrators adopt a manual and intuition-based approach. In this paper, we propose to replace the ad-hoc, manual, intuition-based approach with a more systematic, automated, and analytics-based approach for system monitoring. We propose an adaptive monitoring framework where end-to-end probing-based solutions are used to adapt the at-a-point monitoring tools. We present a systematic framework to use probes for adjusting monitoring levels. We present algorithms to select and analyze probes and to dynamically adapt the monitoring policies based on probe analysis. We demonstrate the effectiveness of the proposed solution using real-world examples as well as simulations.

show abstract

“…Our algorithm is derived from the technique developed in Lakhina et al [16] for wired infrastructure networks. The method of Lakhina et al [16] employs Principal Component Analysis (PCA) [13] for dimension reduction and filtering of observed data and uses Hotelling's t 2 statistics to detect the data points deviating far from the mean traffic conditions. We have evaluated the scheme of Lakhina et al [16] in Hakami et al [8] for the scenario of a WMN testbed deployed in Sydney.…”

Section: Introductionmentioning

confidence: 99%

Real-time detection of traffic anomalies in wireless mesh networks

et al. 2009

View full text Add to dashboard Cite

Anomaly detection is emerging as a necessary component as wireless networks gain popularity. Anomaly detection has been addressed broadly in wired networks and powerful methods have been developed for correct detection of a variety of known attacks and other anomalies. In this paper, we propose a real-time anomaly detection and identification scheme for wireless mesh networks (WMN) using components from previous methods developed for wired networks. Experiments over a WMN testbed show the effectiveness of the proposed scheme in isolating different types of anomalies, such as Denial-ofservice attacks, port scan attacks, etc. Our scheme uses Chi-square statistics and it is based on similar ideas as the scheme presented by Lakhina et al. although it has lower computational complexity. The original method by Lakhina et al. was developed for wired networks and used Principal Component Analysis (PCA) for reducing the dimensions of observed data and Hotelling's t 2 statistics to distinguish between normal and abnormal traffic conditions. However, in our studies we found that dimension reduction is the most computationally intensive process of the scheme. In this paper we propose an alternative way of reducing dimensions using flow variances in a Chi-square test. Experimental results show that the Chi-square test performs similarly well to the PCA-based method at merely a fraction of the computations. Moreover, we propose an automatic identification scheme to pin-point the cause of the detected anomaly and its contribution in terms of additional or lack of traffic. Our results and comparison with other statistical tools show that the Chi-square test and the PCA-based method with identification scheme make powerful tools for real-time detection of various anomalies in an interference prone wireless networking environment.

show abstract

Diagnosing network-wide traffic anomalies

Cited by 395 publications

References 28 publications

On the complexity of Internet traffic dynamics on its topology

On the complexity of Internet traffic dynamics on its topology

Adaptive Monitoring: Application of Probing to Adapt Passive Monitoring

Real-time detection of traffic anomalies in wireless mesh networks

Contact Info

Product

Resources

About