Isconna: Streaming Anomaly Detection with Frequency and Patterns

Abstract

An edge stream is a common form of presentation of dynamic networks. It can evolve with time, with new types of nodes or edges being continuously added. Existing methods for anomaly detection rely on edge occurrence counts or compare pattern snippets found in historical records. In this work, we propose Isconna, which focuses on both the frequency and the pattern of edge records. The burst detection component targets anomalies between individual timestamps, while the pattern detection component highlights anomalies across segments of timestamps. These two components together produce three intermediate scores, which are aggregated into the final anomaly score. Isconna does not actively explore or maintain pattern snippets; it instead measures the consecutive presence and absence of edge records. Isconna is an online algorithm, it does not keep the original information of edge records; only statistical values are maintained in a few count-min sketches (CMS). Isconna's space complexity O(rc) is determined by two user-specific parameters, the size of CMSs. In worst case, Isconna's time complexity can be up to O(rc), but it can be amortized in practice. Experiments show that Isconna outperforms five state-of-the-art frequency-and/or pattern-based baselines on six real-world datasets with up to 20 million edge records.

Distributed Anomaly Detection in Edge Streams using Frequency based Sketch Datastructures

“…A recent study by Liu et al performs streaming anomaly detection with frequency and patterns [12]. Their work proposes online algorithms (termed as Isconna-EO and Isconna-EN) that measure the consecutive presence and absence of edge records.…”