Sketch-Based Anomaly Detection in Streaming Graphs

Abstract

Given a stream of graph edges from a dynamic graph, how can we assign anomaly scores to edges and subgraphs in an online manner, for the purpose of detecting unusual behavior, using constant time and memory? For example, in intrusion detection, existing work seeks to detect either anomalous edges or anomalous subgraphs, but not both. In this paper, we first extend the count-min sketch data structure to a higher-order sketch. This higher-order sketch has the useful property of preserving the dense subgraph structure (dense subgraphs in the input turn into dense submatrices in the data structure). We then propose four online algorithms that utilize this enhanced data structure, which (a) detect both edge and graph anomalies; (b) process each edge and graph in constant memory and constant update time per newly arriving edge, and; (c) outperform state-of-the-art baselines on four real-world datasets. Our method is the first streaming approach that incorporates dense subgraph search to detect graph anomalies in constant memory and time.

Subset Node Anomaly Tracking over Large Dynamic Graphs

“…On the other hand, works [36] on graph-level anomaly detection cannot identify individual node changes but only uncover the global changes in the overall graph structure. Most representative methods leverage tensor factorization [9] and graph sketching [8], detecting the sudden dis/appearance of dense subgraphs. However, we argue that since most anomalies are locally incurred by a few anomalous nodes/edges, the global graph-level anomaly signal may overlook subtle local changes.…”

“…We use Procrustes analysis to find optimal scale, translation and rotation to align6 In experiment, we assume that the number of anomalies is known for each node to calculate the prediction precision 7. We track totally 200 nodes, but 49 nodes have all anomalous edges before the initial snapshot, so we exclude them in precision calculation 8. For Random baseline, we randomly assign anomaly score for each node across snapshots…”