Unsupervised Anomaly Detection With LSTM Neural Networks

Abstract

We investigate anomaly detection in an unsupervised framework and introduce Long Short Term Memory (LSTM) neural network based algorithms. In particular, given variable length data sequences, we first pass these sequences through our LSTM based structure and obtain fixed length sequences. We then find a decision function for our anomaly detectors based on the One Class Support Vector Machines (OC-SVM) and Support Vector Data Description (SVDD) algorithms. As the first time in the literature, we jointly train and optimize the parameters of the LSTM architecture and the OC-SVM (or SVDD) algorithm using highly effective gradient and quadratic programming based training methods. To apply the gradient based training method, we modify the original objective criteria of the OC-SVM and SVDD algorithms, where we prove the convergence of the modified objective criteria to the original criteria. We also provide extensions of our unsupervised formulation to the semisupervised and fully supervised frameworks. Thus, we obtain anomaly detection algorithms that can process variable length data sequences while providing high performance, especially for time series data. Our approach is generic so that we also apply this approach to the Gated Recurrent Unit (GRU) architecture by directly replacing our LSTM based structure with the GRU based structure. In our experiments, we illustrate significant performance gains achieved by our algorithms with respect to the conventional methods.

Anomaly Detection in Beehives using Deep Recurrent Autoencoders

“…time series, deep recurrent autoencoders using long-short-term memory networks (LSTMs) (Goodfellow et al, 2016) have shown great success over conventional methods (e.g. SVM) (Ergen et al, 2017). We adapt this model for this work.…”

Towards Zero Shot Learning of Geometry of Motion Streams and Its Application to Anomaly Recognition

“…Few unsupervised non-visual anomaly recognition arts [35,36,28] have used autoencoders for feature extraction. However, these cannot directly fit on to spatio-temporal data.…”

Recent Advances in Anomaly Detection Methods Applied to Aviation

“…More recently, Ergen et al [59] present a hybrid framework for variable length sequences based on LSTM and the use of either OC-SVM [43] or SVDD [77] as anomaly detectors. The novelty in the approach comes from the fact they jointly optimise the parameters of both LSTM and the anomaly detector by developing specific gradient based training methods.…”