A Family of Joint Sparse PCA Algorithms for Anomaly Localization in Network Data Streams

Abstract

Determining anomalies in data streams that are collected and transformed from various types of networks has recently attracted significant research interest.Principal Component Analysis (PCA) is arguably the most widely applied unsupervised anomaly detection technique for networked data streams due to its simplicity and efficiency. However, none of existing PCA based approaches addresses the problem of identifying the sources that contribute most to the observed anomaly, or anomaly localization. In this paper, we first proposed a novel joint sparse PCA method to perform anomaly detection and localization for network data streams. Our key observation is that we can detect anomalies and localize anomalous sources by identifying a low dimensional abnormal subspace that captures the abnormal behavior of data. To better capture the sources of anomalies, we incorporated the structure of the network stream data in our anomaly localization framework. Also, an extended version of PCA, multidimensional KLE, was introduced to stabilize the localization performance. We performed comprehensive experimental studies on four real-world data sets from different application domains and compared our proposed techniques with several state-of-the-arts. Our experimental studies demonstrate the utility of the proposed methods.

Distributed Real-Time Anomaly Detection in Networked Industrial Sensing Systems

“…The major and minor radius of the ellipseR i,j (t) are computed by using the Principle Component Analysis (PCA) [22], which is a mathematical procedure to convert observations of multiple observers into orthogonal variables called principle components (PCs). These PCs indicate the most representative variances in these observations.…”

Brain EEG Time Series Selection: A Novel Graph-Based Approach for Classification

“…With local and global measurements, the improved Fréchet distance (IFD) is calculated with (4) and (5). sij =λ · norm(δF (tri, trj))…”

“…S tri α(t) , trj β(t) (5) and norm(δF (tri, trj)) ∈ [0, 1] is the normalized value of δF (tri, trj); λ ∈ [0, 1] is the weight for normalized global Fréchet distance while 1 − λ for local tendency. All the similarities of n pairwise connected EEGs calculated by IFD with local and global measurements construct the edge weight of the undirected weighted complete graph.…”

A multi-scale energy detector for anomaly detection in dynamic networks

“…So far, most research has defined anomaly detection as tracking the evolution of clusters over time. Different methods based on the eigenvectors of the modularity or the adjacency matrix have been proposed to detect anomalies and localize them [2,3]. In [4], an optimal FIR filter is designed to integrate the modularity matrices inside a window over the time.…”