Eigenspace-based anomaly detection in computer systems

Idé, Tsuyoshi; Kashima, Hisashi

doi:10.1145/1014052.1014102

Cited by 159 publications

(103 citation statements)

References 23 publications

(28 reference statements)

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…In [22], Sun et al take an information theoretic approach to discover communities and detect changes in dynamic networks in an online manner. In [23], [24], dissimilarity measures of bipartite graphs are proposed by comparing the so-called "behavior" (or "activity") vectors, which is the principal eigenvector of the correlation matrix (or dependency matrix in the latter case). All of these works assume that the graphs that they deal with have the same nodes throughout the whole sequence.…”

Section: Sequence Of Bipartite Graphs: Synthetic Datamentioning

confidence: 99%

Change-point detection in a sequence of bags-of-data

Koshijima¹,

Hino

Motomura

2016

2016 IEEE 32nd International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Abstract-In this paper, the limitation that is prominent in most existing works of change-point detection methods is addressed by proposing a nonparametric, computationally efficient method. The limitation is that most works assume that each data point observed at each time step is a single multi-dimensional vector. However, there are many situations where this does not hold. Therefore, a setting where each observation is a collection of random variables, which we call a bag of data, is considered. After estimating the underlying distribution behind each bag of data and embedding those distributions in a metric space, the change-point score is derived by evaluating how the sequence of distributions is fluctuating in the metric space using a distance-based information estimator. Also, a procedure that adaptively determines when to raise alerts is incorporated by calculating the confidence interval of the change-point score at each time step. This avoids raising false alarms in highly noisy situations and enables detecting changes of various magnitudes. A number of experimental studies and numerical examples are provided to demonstrate the generality and the effectiveness of our approach with both synthetic and real datasets.

show abstract

Section: Sequence Of Bipartite Graphs: Synthetic Datamentioning

confidence: 99%

Change-point detection in a sequence of bags-of-data

Koshijima¹,

Hino

Motomura

2016

2016 IEEE 32nd International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

show abstract

“…In [14], Malliaros et al defined a new graph robustness property based on top k eigen-pairs of the graph, and proposed an algorithm to detect communities and anomalies. Similar mechanism for anomaly detection was used in [11] based on eigen-pairs of dependency matrix of the graph. Ferlez et al [7] proposed a dynamic graph monitoring algorithm based on MDL [3] which can be used to detect the changing communities in the evolving process.…”

Section: Dynamic Graphs Analysismentioning

confidence: 99%

Fast Eigen-Functions Tracking on Dynamic Graphs

Chen¹,

Tong²

2015

Proceedings of the 2015 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Many important graph parameters can be expressed as eigenfunctions of its adjacency matrix. Examples include epidemic threshold, graph robustness, etc. It is often of key importance to accurately monitor these parameters. For example, knowing that Ebola virus has already been brought to the US continent, to avoid the virus from spreading away, it is important to know which emerging connections among related people would cause great reduction on the epidemic threshold of the network. However, most, if not all, of the existing algorithms computing these measures assume that the input graph is static, despite the fact that almost all real graphs are evolving over time. In this paper, we propose two online algorithms to track the eigen-functions of a dynamic graph with linear complexity wrt the number of nodes and number of changed edges in the graph. The key idea is to leverage matrix perturbation theory to efficiently update the top eigen-pairs of the underlying graph without recomputing them from scratch at each time stamp. Experiment results demonstrate that our methods can reach up to 20× speedup with precision more than 80% for fairly long period of time. Keywords: dynamic graph; connectivity; graph spectrum; attribution analysis 1 Introduction Among others, node importance and graph connectivity are of key importance to understand fundamental characteristics of a network (such as social networks, power grid, transportation network, etc). To date, many different parameters of the graph have been invented to measure those properties from different perspective. One most commonly used parameter for node importance estimation is eigenvector centrality [15], which is effective on identifying influential nodes over the whole network. As for connectivity in the graph, important parameters include epidemic threshold ([28, 4, 20]), clustering coefficient [29] and graph robustness ([2, 8, 5]). One interesting observation is that many of those parameters can be calculated or estimated accurately by certain functions of the eigen-pairs of the graph. For example, it has been found that for an arbitrary graph, the tipping point for the dissemination process is controlled by the leading eigenvalue of certain system matrix associated with the graph [28,20]. As for the clustering coefficient computation, instead of doing Most of the algorithms that compute the above parameters/eigen-functions are based on static graphs. However in real world networks, the graph structure keeps evolving over time. It is of great importance to keep track of those measurements since subtle changes on graph structure may lead to sharp changes on the overall properties. For example, in epidemic process, some emerging connections between people may increase the leading eigenvalue a lot, and thus reduce the epidemic threshold, which in turn makes the virus easier to spread through the network. By monitoring related parameters in the network over time, we would be able to know when the change happens and identify its cause/attribution timely. ...

show abstract

“…(3) Online Graph Outlier Detection Algorithms: These include identifying anomalous graph linkage structures [1] from a stream of graphs using reservoir sampling. Spectral methods [14] have been also been used for the same.…”

Section: Related Workmentioning

confidence: 99%

Local Learning for Mining Outlier Subgraphs from Network Datasets

Gupta

Mallya

Roy

et al. 2014

Proceedings of the 2014 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

In the real world, various systems can be modeled using entity-relationship graphs. Given such a graph, one may be interested in identifying suspicious or anomalous subgraphs. Specifically, a user may want to identify suspicious subgraphs matching a query template. A subgraph can be defined as anomalous based on the connectivity structure within itself as well as with its neighborhood. For example for a co-authorship network, given a subgraph containing three authors, one expects all three authors to be say data mining authors. Also, one expects the neighborhood to mostly consist of data mining authors. But a 3-author clique of data mining authors with all theory authors in the neighborhood clearly seems interesting. Similarly, having one of the authors in the clique as a theory author when all other authors (both in the clique and neighborhood) are data mining authors, is also suspicious. Thus, existence of lowprobability links and absence of high-probability links can be a good indicator of subgraph outlierness. The probability of an edge can in turn be modeled based on the weighted similarity between the attribute values of the nodes linked by the edge. We claim that the attribute weights must be learned locally for accurate link existence probability computations. In this paper, we design a system that finds subgraph outliers given a graph and a query by modeling the problem as a linear optimization. Experimental results on several synthetic and real datasets show the effectiveness of the proposed approach in computing interesting outliers.

show abstract

Eigenspace-based anomaly detection in computer systems

Cited by 159 publications

References 23 publications

Change-point detection in a sequence of bags-of-data

Change-point detection in a sequence of bags-of-data

Fast Eigen-Functions Tracking on Dynamic Graphs

Local Learning for Mining Outlier Subgraphs from Network Datasets

Contact Info

Product

Resources

About