Structural analysis of network traffic flows

Abstract

Network traffic arises from the superposition of Origin-Destination (OD) flows. Hence, a thorough understanding of OD flows is essential for modeling network traffic, and for addressing a wide variety of problems including traffic engineering, traffic matrix estimation, capacity planning, forecasting and anomaly detection. However, to date, OD flows have not been closely studied, and there is very little known about their properties.We present the first analysis of complete sets of OD flow timeseries, taken from two different backbone networks (Abilene and Sprint-Europe). Using Principal Component Analysis (PCA), we find that the set of OD flows has small intrinsic dimension. In fact, even in a network with over a hundred OD flows, these flows can be accurately modeled in time using a small number (10 or less) of independent components or dimensions.We also show how to use PCA to systematically decompose the structure of OD flow timeseries into three main constituents: common periodic trends, short-lived bursts, and noise. We provide insight into how the various constitutents contribute to the overall structure of OD flows and explore the extent to which this decomposition varies over time.

Characteristics of origin-destination pair traffic in Funet

“…Lakhina et al [5] present an interesting structural analysis of the traffic demand matrix X. By applying Principal Component Analysis (PCA), they demonstrate that each OD flow (x nk ; n = 1, .…”

Operational experiences with anomaly detection in backbone networks

“…The subspace-method [11,12] analyzes OD-flows (flows with the same origin and destination points of the monitored network). Because of the high dimensional multivariate data structure of that flows, a lower-approximation is needed: Principal Component Analysis (PCA) [17]. This mathematical method captures the most important trends of the explored data (it preserves the significance of the data while reducing its complex initial structure).…”

An Improved Traffic Matrix Decomposition Method with Frequency-Domain Regularization

“…A traffic matrix combines diverse traffic components with distinct temporal properties. Therefore, it is necessary to decompose them efficiently, and this problem is named traffic matrix structural analysis [1]. We presented the traffic matrix decomposition model in [2] and decomposed a traffic matrix into three sub-matrices, which is equivalent to the generalized Robust Principal Component Analysis (RPCA) problem [3].…”