Unsupervised Sequential Outlier Detection With Deep Architectures

Abstract

Unsupervised outlier detection is a vital task and has high impact on a wide variety of applications domains, such as image analysis and video surveillance. It also gains long-standing attentions and has been extensively studied in multiple research areas. Detecting and taking action on outliers as quickly as possible are imperative in order to protect network and related stakeholders or to maintain the reliability of critical systems. However, outlier detection is difficult due to the one class nature and challenges in feature construction. Sequential anomaly detection is even harder with more challenges from temporal correlation in data, as well as the presence of noise and high dimensionality. In this paper, we introduce a novel deep structured framework to solve the challenging sequential outlier detection problem. We use autoencoder models to capture the intrinsic difference between outliers and normal instances and integrate the models to recurrent neural networks that allow the learning to make use of previous context as well as make the learners more robust to warp along the time axis. Furthermore, we propose to use a layerwise training procedure, which significantly simplifies the training procedure and hence helps achieve efficient and scalable training. In addition, we investigate a fine-tuning step to update all parameters set by incorporating the temporal correlation in the sequence. We further apply our proposed models to conduct systematic experiments on five real-world benchmark data sets. Experimental results demonstrate the effectiveness of our model, compared with other state-of-the-art approaches.

Deep Learning for Anomaly Detection

“…AEs are also widely leveraged to detect anomalies in data other than tabular data, such as sequence data [93], graph data [38] and image/video data [164]. In general, there are two types of adaptions of AEs to those complex data.…”

“…These approaches are different from the first type of approaches in that the prediction of representations are wrapped around the low-dimensional representations yielded by AEs. For example, in [93], denoising AE is combined with RNNs to learn normal patterns of multivariate sequence data, in which a denoising AE wtih two hidden layers is first used to learn representations of multidimensional data inputs in each time step and a RNN with a simple single hidden layer is then trained to predict the representations yielded by the denoising AE. A similar approach is also used for detecting acoustic anomalies [99], in which a more complex RNN, bidirectional LSTMs, is used.…”

“…Challenges Targeted. Different types of neural network layers and architectures can be used under the AE framework, allowing us to detect anomalies in high-dimensional data, as well as non-independent data such as attributed graph data [38] and multivariate sequence data [93,99] (CH2). These methods may reduce false positives over traditional methods built upon handcrafted features if the learned representations are more expressive (CH1).…”

Feature Extraction Using Convolutional Neural Networks for Anomaly Detection

“…The autoencoder is an example of an algorithm used for this purpose. Another common anomaly detection approach is to train the reconstruction algorithm, e.g., an autoencoder, and use the outputs of intermediate layers as input features for shallow anomaly detection algorithms [8,9], e.g., one-class support vector machine.…”

“…Aware of this scenario, we propose a new deep learningbased anomaly detection approach. It consists of a hybrid system that combines implicit feature extraction by a deep architecture with a one-class classifier, as already seen in the literature [8,9]. Our contribution focuses on the feature extractor, which is a convolution neural network trained as a regressor to learn a predefined distribution, randomly chosen.…”