Spatio-temporal compressive sensing and internet traffic matrices

ZhangYin,; Roughan, Matthew; WillingerWalter,; QiuLili,

doi:10.1145/1594977.1592600

Cited by 196 publications

(228 citation statements)

References 31 publications

Supporting

Mentioning

225

Contrasting

Order By: Relevance

“…Bharti et al [2] also report on the sparseness of the ITM, and propose methods to infer the invisible elements of the ITM. Our work confirms the sparsity and low effective rank of the ITM seen in previous work [15].…”

Section: Low Effective Ranksupporting

confidence: 91%

See 1 more Smart Citation

Towards a Statistical Characterization of the Interdomain Traffic Matrix

et al. 2012

View full text Add to dashboard Cite

Abstract. Identifying the statistical properties of the Interdomain Traffic Matrix (ITM) is fundamental for Internet techno-economic studies but challenging due to the lack of adequate traffic data. In this work, we utilize a Europe-wide measurement infrastructure deployed at the GÉANT backbone network to examine some important spatial properties of the ITM. In particular, we analyze its sparsity and characterize the distribution of traffic generated by different ASes. Our study reveals that the ITM is sparse and that the traffic sent by an AS can be modeled as the LogNormal or Pareto distribution, depending on whether the corresponding traffic experiences congestion or not. Finally, we show that there exist significant correlations between different ASes mostly due to relatively few highly popular prefixes.

show abstract

Section: Low Effective Ranksupporting

confidence: 91%

“…Some techniques to estimate invisible elements of the ITM (e.g., matrix completion [2,15]) rely on the fact that the ITM has low effective rank.…”

Section: Low Effective Rankmentioning

confidence: 99%

Towards a Statistical Characterization of the Interdomain Traffic Matrix

et al. 2012

View full text Add to dashboard Cite

show abstract

“…As in [29], we use a simple temporal transformation matrix to minimize the change in x between two consecutive days: T = T oeplitz(0, 1, −1), which denotes the Toeplitz matrix with central diagonal given by 1, the first upper diagonal given by -1. That is, …”

Section: Fitting Errormentioning

confidence: 99%

Event detection using customer care calls

Chen

Lee

Duffield

et al. 2013

2013 Proceedings IEEE INFOCOM

Self Cite

View full text Add to dashboard Cite

Abstract-Customer care calls serve as a direct channel for a service provider to learn feedbacks from their customers. They reveal details about the nature and impact of major events and problems observed by customers. By analyzing the customer care calls, a service provider can detect important events to speed up problem resolution. However, automating event detection based on customer care calls poses several significant challenges. First, the relationship between customers' calls and network events is blurred because customers respond to an event in different ways. Second, customer care calls can be labeled inconsistently across agents and across call centers, and a given event naturally give rise to calls spanning a number of categories. Third, many important events cannot be detected by looking at calls in one category. How to aggregate calls from different categories for event detection is important but challenging. Lastly, customer care call records have high dimensions (e.g., thousands of categories in our dataset).In this paper, we propose a systematic method for detecting events in a major cellular network using customer care call data. It consists of three main components: (i) using a regression approach that exploits temporal stability and low-rank properties to automatically learn the relationship between customer calls and major events, (ii) reducing the number of unknowns by clustering call categories and using L1 norm minimization to identify important categories, and (iii) employing multiple classifiers to enhance the robustness against noise and different response time. For the detected events, we leverage Twitter social media to summarize them and to locate the impacted regions. We show the effectiveness of our approach using data from a large cellular service provider in the US.

show abstract

“…These studies have implications for effective network management and capacity planning, and provide a foundation for our work. A related topic is traffic matrix estimation, which seeks to identify traffic volumes for origin-destination flows through a network based on partial information (e.g., [25], [29], [34]). A motivation for our study is to develop a tool that could be used to systematically assess and evaluate traffic matrix estimation techniques.…”

Section: Related Workmentioning

confidence: 99%

“…Automated detection of anomalies in computer networks has been of interest for a number of years, e.g., [16], [18]- [20], [33], [34]. However, quantitative assessment of anomaly detection algorithms remains challenging.…”

Section: Application: Anomaly Detectionmentioning

confidence: 99%

Efficient network-wide flow record generation

Sommers

Bowden

Eriksson³

et al. 2011

2011 Proceedings IEEE INFOCOM

Self Cite

View full text Add to dashboard Cite

Abstract-Experiments on diverse topics such as network measurement, management and security are routinely conducted using empirical flow export traces. However, the availability of empirical flow traces from operational networks is limited and frequently comes with significant restrictions. Furthermore, empirical traces typically lack critical meta-data (e.g., labeled anomalies) which reduce their utility in certain contexts. In this paper, we describe fs: a first-of-its-kind tool for automatically generating representative flow export records as well as basic SNMP-like router interface counts. fs generates measurements for a target network topology with specified traffic characteristics. The resulting records for each router in the topology have byte, packet and flow characteristics that are representative of what would be seen in a live network. fs also includes the ability to inject different types of anomalous events that have precisely defined characteristics, thereby enabling evaluation of proposed attack and anomaly detection methods. We validate fs by comparing it with the ns-2 simulator, which targets accurate recreation of packet-level dynamics in small network topologies. We show that data generated by fs are virtually identical to what are generated by ns-2, except over small time scales (below 1 second). We also show that fs is highly efficient, thus enabling test sets to be created for large topologies. Finally, we demonstrate the utility of fs through an assessment of anomaly detection algorithms, highlighting the need for flexible, scalable generation of network-wide measurement data with known ground truth.

show abstract

Spatio-temporal compressive sensing and internet traffic matrices

Cited by 196 publications

References 31 publications

Towards a Statistical Characterization of the Interdomain Traffic Matrix

Towards a Statistical Characterization of the Interdomain Traffic Matrix

Event detection using customer care calls

Efficient network-wide flow record generation

Contact Info

Product

Resources

About