Spatial neighborhood based anomaly detection in sensor datasets

Abstract

Success of anomaly detection, similar to other spatial data mining techniques, relies on neighborhood definition. In this paper, we argue that the anomalous behavior of spatial objects in a neighborhood can be truly captured when both (a) spatial autocorrelation (similar behavior of nearby objects due to proximity) and (b) spatial heterogeneity (distinct behavior of nearby objects due to difference in the underlying processes in the region) are taken into consideration for the neighborhood definition. Our approach begins by generating micro neighborhoods around spatial objects encompassing all the information about a spatial object. We selectively merge these based on spatial relationships accounting for autocorrelation and inferential relationships accounting for heterogeneity, forming macro neighborhoods. In such neighborhoods, we then identify (i) spatio-temporal outliers, where individual sensor readings are anomalous, (ii) spatial outliers, where the entire sensor is an anomaly, and (iii) spatio-temporally coalesced outliers, where a group of spatio-temporal outliers in the macro neighborhood are separated by a small time lag indicating the traversal Responsible editor: Sanjay Chawla. 123 222 V. P. Janeja et al.of the anomaly. We demonstrate the effectiveness of our approach in neighborhood formation and anomaly detection with experimental results in (i) water monitoring and (ii) highway traffic monitoring sensor datasets. We also compare the results of our approach with an existing approach for spatial anomaly detection.

There and back again: Outlier detection between statistical reasoning and data mining algorithms

“…As data mining often has to deal with data of particular characteristics, many specialized outlier detection methods have been proposed. Fundamental challenges of high‐dimensional data for outlier detection have been discussed by Zimek et al (), and various methods dedicated to outlier detection in high‐dimensional data have been proposed (Dang, Assent, Ng, Zimek, & Schubert, ; de Vries, Chawla, & Houle, ; Keller, Müller, & Böhm, ; Kriegel, Kröger, et al, ; Kriegel, Kröger, Schubert, & Zimek, , ; Müller, Schiffer, & Seidl, , ; Nguyen, Gopalkrishnan, & Assent, ; Pham & Pagh, ). Another large subtopic is outlier detection in spatial data, trajectory data, or some other notion of separating context and indicator variables (Chawla & Sun, ; Hayes & Capretz, ; Janeja, Adam, Atluri, & Vaidya, ; Kou, Lu, & Chen, ; Leach, Sparks, & Robertson, ; Lee, Han, & Li, ; Liang & Parthasarathy, ; Liu, Lu, & Chen, ; Lu, Chen, & Kou, ; Shekhar, Lu, & Zhang, ; Song, Wu, Jermaine, & Ranka, ; Sun & Chawla, ). The spatial neighborhood can be interpreted as a special case of locality for local outlier detection.…”

Detecting spatio-temporal outliers in crowdsourced bathymetry data

“…Other statistical methods use measures of spatial and temporal autocorrelation to detect outliers in sensor networks [9]. A number of approaches use a neighborhood-based method [10,11,12] where a graph data structure is used to model relationships between adjacent spatial measurements and a pruning algorithm is used to find outliers in the connected graph. Clustering-based methods [13] are generally focused on minimizing the distance between measurements taken at adjacent locations and using the clustering result to find outlying areas.…”

Identifying patterns in spatial information: A survey of methods

“…In the context of transportation networks, researchers proposed similar ST outlier patterns for identifying traffic accidents known as anomalous window discovery. [123][124][125] Teleconnected flow anomalies: An additional pattern that utilizes FAs is teleconnected patterns. 126 A teleconnection represents a strong interaction between paired events that are spatially distant from each other.…”

“…However, further research is needed to discover other types of patterns within this environment. In the context of transportation networks, researchers proposed similar ST outlier patterns for identifying traffic accidents known as anomalous window discovery 123–125…”