dLSTM: a new approach for anomaly detection using deep learning with delayed prediction

Abstract

In this paper, we propose delayed Long Short-Term Memory (dLSTM), an anomaly detection method for time-series data. We first build a predictive model from normal (non-anomalous) training data, then perform anomaly detection based on the prediction error for observed data. However, there are multiple states in the waveforms of normal data, which may lower prediction accuracy. To deal with this problem, we utilize multiple prediction models based on LSTM for anomaly detection. In this scheme, the prediction accuracy strongly depends on the method of selecting a proper predictive model from multiple possible models. We propose a novel method to determine the proper predictive model for anomaly detection. Our approach provides multiple predicted value candidates in advance and selects the one that is closest to the measured value. We delay the model selection until the corresponding measured values are acquired. Using this concept for anomaly detection, dLSTM selects the proper predictive model to enhance prediction accuracy. In our experimental evaluation using real and artificial data, dLSTM detects anomalies more accurately than methods in comparison.

A Review on Machine Learning Approaches for Network Malicious Behavior Detection in Emerging Technologies

“…Moreover, for any future works, the authors suggest the improvement of the method by using RNN and long short-term memory (LSTM) models [ 91 ]. Maya et al [ 92 ] proposed an RNN model based on delayed long short-term memory (dLSTM) for network malicious pattern detection on the time-series data. In the first step, a predictive model was generated from normal traffic instances, then identified malicious patterns based on the prediction error for observed data.…”

The Deployment of a LoRaWAN-Based IoT Air Quality Sensor Network for Public Good

“…Several of these projects have shown success when using random forest models to calibrate their sensors and reinforce the improvements to accuracy possible when sensing multiple air quality metrics simultaneously [18]. Other projects report success in the application of LSTM Neural Network approaches to calibrate their sensors [19]- [21].…”

Introduction to the special issue on Big Data, IoT Streams and Heterogeneous Source Mining

“…to Process Models using Prefix-Alignments [4] provides a novel approach to incrementally compute prefixalignments, thus paving the way for real-time online conformance checking. -dLSTM: a new approach for anomaly detection using deep learning with delayed prediction [5] develops a method based on several LSTMs that can detect anomalies in time-series data and dynamically select the most appropriate model for prediction.…”