DenseAlert

Abstract

Consider a stream of retweet events -how can we spot fraudulent lock-step behavior in such multi-aspect data (i.e., tensors) evolving over time? Can we detect it in real time, with an accuracy guarantee? Past studies have shown that dense subtensors tend to indicate anomalous or even fraudulent behavior in many tensor data, including social media, Wikipedia, and TCP dumps. us, several algorithms have been proposed for detecting dense subtensors rapidly and accurately. However, existing algorithms assume that tensors are static, while many real-world tensors, including those mentioned above, evolve over time.We propose D S , an incremental algorithm that maintains and updates a dense subtensor in a tensor stream (i.e., a sequence of changes in a tensor), and D A , an incremental algorithm spo ing the sudden appearances of dense subtensors. Our algorithms are: (1) Fast and 'any time': updates by our algorithms are up to a million times faster than the fastest batch algorithms, (2) Provably accurate: our algorithms guarantee a lower bound on the density of the subtensor they maintain, and (3) E ective: our D A successfully spots anomalies in realworld tensors, especially those overlooked by existing algorithms.

Fast and Multi-aspect Mining of Complex Time-stamped Event Streams

“…Typical anomaly detection methods [15,25,31,61,81], such as local outlier factor (LOF) [24] and tree-based approaches [34,56], can be used in event tensors by converting multiple attributes to numerical ones. [20,45,62,79,80] use a stream of multi-aspect records as input. MemStream [21] can learn dynamically changing trends to handle time-varying data distribution known as concept drift [27,35,57].…”

“…Given a large, online stream of time-stamped events, how can we statistically summarize all the event streams and find important patterns, rules, and anomalies? Time-stamped event data are generated and collected by many real applications [10,17,29,84], including online marketing analytics [52,75], social network/location-based services [28,71], and cybersecurity systems [19,80], with increasingly larger sizes and faster rates of transactions. For example, an online shopping service could generate millions of logging entries every second, with rich information about items and users.…”

A comprehensive survey of anomaly detection techniques for high dimensional big data

“…A sophisticated approach handles the number of increasing dimensions without affecting accuracy in either situation. High-dimensional data is often referred to as multi-dimensional or multi-aspect or multi-modal data throughout the literature [43,44]. Highly detailed data from diverse platforms such as the web, social media is typically high-dimensional in nature.…”