Adaptive Sampling and Statistical Inference for Anomaly Detection

Huang, Tingshan

doi:10.17918/etd-6647

Cited by 2 publications

(1 citation statement)

References 84 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where x max is the column vector with unit norm that achieves the maximum matrix norm. Similarly, using the definition of the matrix norm again: A more detailed proof with all intermediate steps is provided in [37]. The value of sin(θ k,k (A, B)), defined earlier as T k,k (A, B) , is the first singular value of T k,k (A, B) and also the square root of the largest eigenvalue of T ′ k,k (A, B)T k,k (A, B).…”

Section: A Proof Of Theoremmentioning

confidence: 99%

A New Approach to Dimensionality Reduction for Anomaly Detection in Data Traffic

Huang

Sethu

Kandasamy

2016

IEEE Trans. Netw. Serv. Manage.

Self Cite

View full text Add to dashboard Cite

The monitoring and management of high-volume feature-rich traffic in large networks offers significant challenges in storage, transmission and computational costs. The predominant approach to reducing these costs is based on performing a linear mapping of the data to a low-dimensional subspace such that a certain large percentage of the variance in the data is preserved in the low-dimensional representation. This variance-based subspace approach to dimensionality reduction forces a fixed choice of the number of dimensions, is not responsive to real-time shifts in observed traffic patterns, and is vulnerable to normal traffic spoofing. Based on theoretical insights proved in this paper, we propose a new distance-based approach to dimensionality reduction motivated by the fact that the real-time structural differences between the covariance matrices of the observed and the normal traffic is more relevant to anomaly detection than the structure of the training data alone. Our approach, called the distance-based subspace method, allows a different number of reduced dimensions in different time windows and arrives at only the number of dimensions necessary for effective anomaly detection. We present centralized and distributed versions of our algorithm and, using simulation on real traffic traces, demonstrate the qualitative and quantitative advantages of the distance-based subspace approach.

show abstract

Section: A Proof Of Theoremmentioning

confidence: 99%