Interactive outlier exploration in big data streams

Abstract

We demonstrate our VSOutlier system for supporting interactive exploration of outliers in big data streams. VSOutlier not only supports a rich variety of outlier types supported by innovative and efficient outlier detection strategies, but also provides a rich set of interactive interfaces to explore outliers in real time. Using the stock transactions dataset from the US stock market and the moving objects dataset from MITRE, we demonstrate that the VSOutlier system enables analysts to more efficiently identify, understand, and respond to phenomena of interest in near real-time even when applied to high volume streams.

Large dataset summarization with automatic parameter optimization and parallel processing for local outlier detection

“…5,[23][24][25][26][27][28] There has also been some research work on interactive outlier detection, which introduces user-friendly interactive and visualization features to assist outlier detection. 6,[29][30][31][32][33] A comprehensive survey of recent outlier detection techniques has been conducted in other works. [34][35][36] It is very important to point out that our proposed method is fundamentally different from all the existing outlier detection in terms of its basic paradigm.…”

Clustering stream data by exploring the evolution of density mountain

“…Similarly, in DP clustering we aim to minimize the average of relative intra-dependent-distance c:δc≤τ δc /δ m and at the same time maximize the average of relative interdependent-distance c:δc>τ δc/δ n where m is the number of intra-cluster-cells m = |{c : δc ≤ τ }|, n is the number of inter-cluster-cells n = |{c : δc > τ }|, δ is the average dependent distance δ = c δc n+m . With regard to time information, we propose the following evaluation function for stream clustering and aim to minimize it 5 .…”

“…Discovering of the patterns hidden in streams is substantial and essential for understanding and further utilizing these data, and there are large number of efforts contribute it, such as [3, 5,13]. Take the news recommendation system as an example.…”

Scalable validation of industrial equipment using a functional DSMS

“…Because data streams are online and dynamic, outlier detection in the stream context becomes fundamentally different than regular outlier detection, which often done in a store-and-process fashion. In Sadik and Gruenwald (2014) previous work on stream outlier detection is categorized into four major classes: (i) outlier detection over sliding windows (Angiulli and Fassetti 2010;Cao et al 2014;Subramaniam et al 2006;Yang et al 2009), (ii) auto-regression (Shuai et al 2008) , (iii) data stream clustering (Yang et al 2009), and (iv) statistical density functions over data stream elements (Huang et al 2012;Subramaniam et al 2006;Zhang et al 2008). Because SVALI makes no difference between regular data types and stream objects, anomaly detection using SVALI's built in validation functions falls into the first category.…”