Future Frame Prediction Using Convolutional VRNN for Anomaly Detection

Abstract

Anomaly detection in videos aims at reporting anything that does not conform the normal behaviour or distribution. However, due to the sparsity of abnormal video clips in real life, collecting annotated data for supervised learning is exceptionally cumbersome. Inspired by the practicability of generative models for semi-supervised learning, we propose a novel sequential generative model based on variational autoencoder (VAE) for future frame prediction with convolutional LSTM (ConvLSTM). To the best of our knowledge, this is the first work that considers temporal information in future frame prediction based anomaly detection framework from the model perspective. Our experiments demonstrate that our approach is superior to the state-of-the-art methods on three benchmark datasets.

Learning Not to Reconstruct Anomalies

“…Ped2 [22] Ave [26] Sh [30] Miscellaneous AbnormalGAN [39] 93.5% --Smeureanu et al [41] -84.6% -AMDN [49,50] 90.8% --STAN [19] 96.5% 87.2% -MC2ST [25] 87.5% 84.4% -Ionescu et al [13] -88.9% -BMAN [20] 96.6% 90.0% 76.2% AMC [34] 96.2% 86.9% -Vu et al [45] 99.21% 71.54% -DeepOC [47] -86.6% -TAM-Net [14] 98.1% 78.3% -LSA [1] 95.4% -72.5% Ramachandra et al [38] 94.0% 87.2% -Tang et al [44] 96.3% 85.1% 73.0% Wang et al [46] -87.0% 79.3% OGNet [54] 98.1% --Conv-VRNN [27] 96.06% 85.78% -Chang et al [3] 96.5% 86.0% 73.3%…”

“…In contrast, our approach encourages AEs to produce unconstrained reconstructions for normal inputs while limiting the reconstructions for anomalous inputs, thus producing more discriminative anomaly scores. Non-Reconstruction Methods: Several researchers adopt different schemes for OCC based anomaly detection: focusing only on objects by utilizing object detectors in the frameworks [6,7,8,11,12,43,52]; predicting future frames from the past few consecutive frames with the intuition that it is difficult to predict unseen anomalous data [5,24,27,28,35]; or incorporating adversarial components [14,19,20,24,39,45]. Our approach is different as we do not utilize any additional component and solely rely on the reconstruction based AEs.…”

In-painting Radiography Images for Unsupervised Anomaly Detection

“…parts of the input image are masked out and the model is trained to recover the missing parts in a self-supervised way [19,37,47,52,78]. There are also plenty of works on detecting anomalies in video sequences [10,18,41,42]. Recently, Bergmann et al [4] and Salehi et al [55] proposed student-teacher networks similar to ours, whereas our method utilizes such a structure to distillate input-aware features only, and the teacher network is completely disabled during inference.…”

Learning Of Linear Video Prediction Models In A Multi-Modal Framework For Anomaly Detection

“…Consequently, models can be learned to predict the evolution of the state of these different sensors. This prediction capability has a variety of applications, among which the detection of anomalies is a major one [3][4][5]. Information from different sensors can be additionally fused in a variety of ways.…”