Compressive Sensing of Internet Traffic Matrices using CUR Decomposition

Kumar, Awnish; Saradhi, V. Vijaya; Tamarapalli, Venkatesh

doi:10.1145/3154273.3154315

Cited by 6 publications

(6 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A disadvantage of their model is that it uses the Moore-Penrose pseudoinverse hence requiring an additional step to suppress possible negative OD flows in the initial estimate of the model. In [11], Kumar et al proposed a traffic estimation model based on a relatively new unconstrained dimensionality reduction technique known as CUR [12]. Its key benefits over SVD are mainly computational efficiency and the interpretability of the low-rank factors that are directly derived from the given data.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Structured Nonnegative Matrix Factorization for Traffic Flow Estimation of Large Cloud Networks

Atif,

Gillis,

Qazi

et al. 2021

Preprint

View full text Add to dashboard Cite

Network traffic matrix estimation is an ill-posed linear inverse problem: it requires to estimate the unobservable origin destination traffic flows, X, given the observable link traffic flows, Y, and a binary routing matrix, A, which are such that Y = AX. This is a challenging but vital problem as accurate estimation of OD flows is required for several network management tasks. In this paper, we propose a novel model for the network traffic matrix estimation problem which maps high-dimension OD flows to low-dimension latent flows with the following three constraints: (1) nonnegativity constraint on the estimated OD flows, (2) autoregression constraint that enables the proposed model to effectively capture temporal patterns of the OD flows, and (3) orthogonality constraint that ensures the mapping between low-dimensional latent flows and the corresponding link flows to be distance preserving. The parameters of the proposed model are estimated with a training algorithm based on Nesterov accelerated gradient and generally shows fast convergence. We validate the proposed traffic flow estimation model on two real backbone IP network datasets, namely Internet2 and G ÉANT. Empirical results show that the proposed model outperforms the state-of-the-art models not only in terms of tracking the individual OD flows but also in terms of standard performance metrics. The proposed model is also found to be highly scalable compared to the existing state-of-the-art approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Recently, Qazi et al [13] proposed to use the demand matrix and the traffic probability matrix pair to transform the ill-posed traffic estimation problem into an equivalent well-posed problem. However, the two proposed methods [11,13] suffer from the limitations similar to that of [9] due to the use of the Moore-Penrose pseudoinverse.…”

Section: Related Workmentioning

confidence: 99%

Structured Nonnegative Matrix Factorization for Traffic Flow Estimation of Large Cloud Networks

Atif,

Gillis,

Qazi

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Special terms have been coined in research literature for estimation of network parameters like network delays [4][5] and traffic volumes [1][2][3]. These techniques have been branded into Kriging [10][11][12], Cartography [13], Tomography [1][2][3] and Compressed Sensing [14][15][16][17][18]. Technically, all of three approaches can be divided into space based, time based and spatio-temporal methods [19].…”

Section: Related Workmentioning

confidence: 99%

“…For GEANT good OD tracking efficiency is possible when at least 65 of 71 paths are monitored. Now, we compare the performance of our proposed scheme (CS-DME) with three other state-of-the-art Compressed Sensing (CS) schemes proposed in recent research literature; Principal Component Analysis (PCA) proposed by Soule et al [18], CUR Decomposition (CUR) proposed by Kumar et al [17] and the Probability based Model Estimation, proposed by Tian et al [33] Abilene Network which we refer to as CS-PCA, CS-CUR and CS-PME respectively in the remainder of this paper. The first two schemes (CS-PCA & CS-CUR) are state of the art compressed sensing schemes which differ from our scheme in the fact that although they allow for the measurement matrix to be compressed efficiently using k principal components they do not allow for a dynamic measurement matrix in the model such as one based on Traffic Demands suitable for modern software defined networks.…”

Section: Performance Evaluationmentioning

confidence: 99%

A Novel Compressed Sensing Technique for Traffic Matrix Estimation of Software Defined Cloud Networks

Qazi¹,

Atif²,

Kadri³

2018

KSII TIIS

View full text Add to dashboard Cite

Traffic Matrix estimation has always caught attention from researchers for better network management and future planning. With the advent of high traffic loads due to Cloud Computing platforms and Software Defined Networking based tunable routing and traffic management algorithms on the Internet, it is more necessary as ever to be able to predict current and future traffic volumes on the network. For large networks such origin-destination traffic prediction problem takes the form of a large under-constrained and under-determined system of equations with a dynamic measurement matrix. Previously, the researchers had relied on the assumption that the measurement (routing) matrix is stationary due to which the schemes are not suitable for modern software defined networks. In this work, we present our Compressed Sensing with Dynamic Model Estimation (CS-DME) architecture suitable for modern software defined networks. Our main contributions are: (1) we formulate an approach in which measurement matrix in the compressed sensing scheme can be accurately and dynamically estimated through a reformulation of the problem based on traffic demands. (2) We show that the problem formulation using a dynamic measurement matrix based on instantaneous traffic demands may be used instead of a stationary binary routing matrix which is more suitable to modern Software Defined Networks that are constantly evolving in terms of routing by inspection of its Eigen Spectrum using two real world datasets. (3) We also show that linking this compressed measurement matrix dynamically with the measured parameters can lead to acceptable estimation of Origin Destination (OD) Traffic flows with marginally poor results with other state-of-art schemes relying on fixed measurement matrices. (4) Furthermore, using this compressed reformulated problem, a new strategy for selection of vantage points for most efficient traffic matrix estimation is also presented through a secondary compression technique based on subset of link measurements. Experimental evaluation of proposed technique using real world datasets Abilene and GEANT shows that the technique is practical to be used in modern software defined networks. Further, the performance of the scheme is compared with recent state of the art techniques proposed in research literature.

show abstract

“…The high-level applications such as anomaly detection require complete TM, and therefore, many studies have been proposed to recover the missing elements of TM from the partial observations of it. Such a method is termed TM completion and some studies have been proposed that use matrix or tensor completion methods [8][9][10] and neural network based methods [11][12][13][14][15].…”

Section: Introductionmentioning

confidence: 99%

Traffic matrix completion by weighted tensor nuclear norm minimization and time slicing

Miyata

2024

NOLTA

View full text Add to dashboard Cite

A traffic matrix (TM) plays an essential role in many network analysis tasks. Since obtaining the full TM by direct observation is challenging, many studies have focused on recovering the TM from the partial observation. These existing methods, which achieve high recovery accuracy using the spatio-temporal characteristics of TM, are computationally expensive and/or require prior training. We propose a new TM completion method based on a nonlinear, nonconvex optimization for a weighted tensor nuclear norm minimization with tensor construction based on the intrinsic periodicity of TM. Our tensor construction method does not require complicated spatio-temporal characteristic estimation and prior training. The experimental results on two real-world traffic data and various data-missing scenarios show that the proposed method can achieve the comparable recovery capability as the conventional methods with a significantly simpler problem formulation.

show abstract

Compressive Sensing of Internet Traffic Matrices using CUR Decomposition

Cited by 6 publications

References 22 publications

Structured Nonnegative Matrix Factorization for Traffic Flow Estimation of Large Cloud Networks

Structured Nonnegative Matrix Factorization for Traffic Flow Estimation of Large Cloud Networks

A Novel Compressed Sensing Technique for Traffic Matrix Estimation of Software Defined Cloud Networks

Traffic matrix completion by weighted tensor nuclear norm minimization and time slicing

Contact Info

Product

Resources

About