Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection

Abstract

Deep autoencoder has been extensively used for anomaly detection. Training on the normal data, the autoencoder is expected to produce higher reconstruction error for the abnormal inputs than the normal ones, which is adopted as a criterion for identifying anomalies. However, this assumption does not always hold in practice. It has been observed that sometimes the autoencoder "generalizes" so well that it can also reconstruct anomalies well, leading to the miss detection of anomalies. To mitigate this drawback for autoencoder based anomaly detector, we propose to augment the autoencoder with a memory module and develop an improved autoencoder called memory-augmented autoencoder, i.e. MemAE. Given an input, MemAE firstly obtains the encoding from the encoder and then uses it as a query to retrieve the most relevant memory items for reconstruction. At the training stage, the memory contents are updated and are encouraged to represent the prototypical elements of the normal data. At the test stage, the learned memory will be fixed, and the reconstruction is obtained from a few selected memory records of the normal data. The reconstruction will thus tend to be close to a normal sample. Thus the reconstructed errors on anomalies will be strengthened for anomaly detection. MemAE is free of assumptions on the data type and thus general to be applied to different tasks. Experiments on various datasets prove the excellent generalization and high effectiveness of the proposed MemAE.

Dual Discriminator Generative Adversarial Network for Video Anomaly Detection

“…StackRNN [25] is an approach that combines a deep neural network and sparse coding. MemAE [30] memorizes normality to detect anomalies via a memoryaugmented deep auto-encoder. LSA [31] adds latent space autoregression to an auto-encoder for frame reconstruction.…”

PyAnomaly: A Pytorch-based Toolkit for Video Anomaly Detection

“…First of all, we will introduce the implemented state-of-the-art methods in the PyAnomaly. In this paper, we classify the existed video anomaly detection methods into four categories: 1) classification [17]; 2) reconstruction [5,8,15]; 3) prediction [12]; and 4) reconstruction + prediction [18,21]. In the early stage, some researchers directly use the action classification methods for video anomaly detection.…”

“…The existed methods which only use the normal data can be further classified into reconstruction and prediction categories. The reconstruction-based methods are proposed relying on an assumption that the anomalies cannot be accurately represented and reconstructed by a model learned only on normal data [5,8,15]. While the prediction methods try to predict future video frames based on previous video frames, and compare the prediction with the real frame for anomaly detection [12].…”

“…However, video anomaly detection is very challenging because: 1) the difficulty of defining the anomaly in different scenarios, and 2) the complex and changeable environments. Recently, researchers propose many promising solutions for video anomaly detection based on various complex video analysis modules [5,6,8,11,12,15,[17][18][19]21]. However, these complex methods and modules are implemented in different deep learning frameworks and settings, which cause the predicament of reproducing these methods, especially ensuring their performance.…”

Neural memory plasticity for medical anomaly detection

“…Recently, a neural memory network based approach for anomaly detection is proposed in [39] where the authors try to memorise the patterns within the normal data in order to detect abnormal instances. However, this approach is quite distinct from the proposed approach as we are learning our memory model from both normal and abnormal data and as such the memory learns to store distinctive characteristics from both normal and abnormal data streams.…”