Foreground Segmentation for Anomaly Detection in Surveillance Videos Using Deep Residual Networks

Abstract

Efficient anomaly detection in surveillance videos across diverse environments represents a major challenge in Computer Vision. This paper proposes a background subtraction approach based on the recent deep learning framework of residual neural networks that is capable of detecting multiple objects of different sizes by pixel-wise foreground segmentation. The proposed algorithm takes as input a reference (anomalyfree) and a target frame, both temporally aligned, and outputs a segmentation map of same spatial resolution where the highlighted pixels denoting the detected anomalies, which should be all the elements not present in the reference frame. Furthermore, we analyze the benefits of different reconstruction methods to the restore original image resolution and demonstrate the improvement of residual architectures over the smaller and simpler models proposed by previous similar works. Experiments show competitive performance in the tested dataset, as well as real-time processing capability.

Learning multi-scale features for foreground segmentation

“…However, this approach is computationally inefficient and may cause overfitting due to redundant pixels, loss of higher context information within patches, and requires large number of patches in training. [16,23,24,27,28] approach the problem by using whole resolution images to the network to predict foreground masks. Some methods take advantages of temporal data [24,27], some methods train the networks by combining image frames with the generated background models [25][26][27][28].…”

Deep neural network concepts for background subtraction:A systematic review and comparative evaluation

“…Cinelli [37] proposed a similar method than Braham and Droogenbroeck [22] by exploring the advantages of Fully Convolutional Neural Networks (FCNNs) [119] to diminish the computational requirements. FCNN use convolutional layer to replace the fully connected layer in traditional convolution networks, which can avoid the disadvantages caused by fully connection layer.…”

“…From this study, the best models on the CDnet 2014 dataset [203] are the 32-layer CIFAR-derived dilated network and the pre-trained 34layer ILSVRC-based dilated model adapted by direct substitution. But, Cinelli [37] only provided visual results without F-measure.…”

“…Since this work, even larger and deeper networks have been trained with the progress made by the storage for Big Data and by the GPUs for deep learning. For the field of background/foreground separation, DNNs were applied with success 1) for background generation [67,151,211,212,213], 2) for background subtraction [4,13,22,37,113], 3) foreground detection enhancement [220], 4) for ground-truth generation [204], and 5) for learned deep spatial features [108,143,166,167,222]. More practically, Restricted Boltzman Machine (RBM) was employed by Guo and Qi [67] and Xu et al [211] for background generation in order to further achieve moving object detection by background subtraction.…”

“…In a similar man-ner, Xu et al [212,213] used deep auto-encoder networks to achieve the same task while Qu et al [151] used context-encoder for background initialization. In another approach, Convolutional Neural Networks (CNNs) were employed for background subtraction by Braham and Droogenbroeck [22], Bautista et al [13], and Cinelli [37]. Other authors employed improved CNNs like Cascaded CNNs [204], deep CNNs [4], structured CNNs [113] and two stage CNNs [226].…”