Abnormal Behavior Detection in Uncrowded Videos with Two-Stream 3D Convolutional Neural Networks

Abstract

The increasing demand for surveillance systems has resulted in an unprecedented rise in the volume of video data being generated daily. The volume and frequency of the generation of video streams make it both impractical as well as inefficient to manually monitor them to keep track of abnormal events as they occur infrequently. To alleviate these difficulties through intelligent surveillance systems, several vision-based methods have appeared in the literature to detect abnormal events or behaviors. In this area, convolutional neural networks (CNNs) have also been frequently applied due to their prevalence in the related domain of general action recognition and classification. Although the existing approaches have achieved high detection rates for specific abnormal behaviors, more inclusive methods are expected. This paper presents a CNN-based approach that efficiently detects and classifies if a video involves the abnormal human behaviors of falling, loitering, and violence within uncrowded scenes. The approach implements a two-stream architecture using two separate 3D CNNs to accept a video and an optical flow stream as input to enhance the prediction performance. After applying transfer learning, the model was trained on a specialized dataset corresponding to each abnormal behavior. The experiments have shown that the proposed approach can detect falling, loitering, and violence with an accuracy of up to 99%, 97%, and 98%, respectively. The model achieved state-of-the-art results and outperformed the existing approaches.

LightAnomalyNet: A Lightweight Framework for Efficient Abnormal Behavior Detection

“…The method could accurately classify different suspicious behaviors found in the mini-drone video dataset. Mehmood [10] studied the specifics of motion patterns involved in three abnormal behaviors, i.e., falling, loitering, and violence, and developed a new dataset by selecting videos pertaining to those patterns from public datasets. A two-stream inflated 3D CNN model pre-trained on the Kinetics dataset was then fine-tuned on the newly developed dataset for the detection of the three anomalies.…”

“…So, for example, the acts of balancing attempts made by a person falling have much in common with the patterns commonly found in suspicious and violent behaviors. Therefore, the solutions targeting this problem mostly aim at providing inclusive methods for detecting multiple anomalies [3,10,11], customizing datasets to learn specific features of targeted behaviors [10], and using advanced techniques of learning the motion patterns [12][13][14], often by incorporating both spatial and temporal features. The second difficulty involves the computational complexity of behavior representation and detection algorithms, resulting in the high expense of computing resources, thus impeding their utilization in many real-world scenarios.…”

“…By focusing on more than one anomaly, as mentioned above, the study combines the classification of various commonly found abnormal behaviors in an uncrowded scene. As reported by previous studies such as [10], because of the challenges resulting from the essential similarities in the acts of various abnormal behaviors, the efficient joint detection of different anomalies (leading to an accurate classification as normal and abnormal behavior) is an interesting problem, and hence a notable contribution of the current study.…”

Spatio-Temporal Deep Learning-Based Methods for Defect Detection: An Industrial Application Study Case

“…However, even though optical flow is widely used to describe motion information, a 3D convolutional kernel may improve the extraction of temporal patterns. This is the main focus in [25], whose author proposed a two-stream 3D-CNN architecture to detect anomalous events in videos. This architecture is also a two-stream CNN approach composed of a network to handle spatial information and another one to handle temporal information.…”

Research on video anomaly detection with variational auto-encoder based on multi-level memory enhancement

“…In order to constrain the generalization ability of the autoencoder as much as possible, a common solution is to use a dual-stream autoencoder to reconstruct the video image and the corresponding optical flow image. The gap between the images can be used to identify whether there is abnormal behavior in the video frame [3][4][5][6][7] ; another solution is to add a memory module in the autoencoder to enhance the normality in the extracted features. Weights, suppress the expression of abnormal weights, so as to achieve the purpose of constraining the generalization of the autoencoder 8,9 .…”