A New Approach for Abnormal Human Activities Recognition Based on ConvLSTM Architecture

Abstract

Recognizing various abnormal human activities from video is very challenging. This problem is also greatly influenced by the lack of datasets containing various abnormal human activities. The available datasets contain various human activities, but only a few of them contain non-standard human behavior such as theft, harassment, etc. There are datasets such as KTH that focus on abnormal activities such as sudden behavioral changes, as well as on various changes in interpersonal interactions. The UCF-crime dataset contains categories such as fighting, abuse, explosions, robberies, etc. However, this dataset is very time consuming. The events in the videos occur in a few seconds. This may affect the overall results of the neural networks that are used to detect the incident. In this article, we create a dataset that deals with abnormal activities, containing categories such as Begging, Drunkenness, Fight, Harassment, Hijack, Knife Hazard, Normal Videos, Pollution, Property Damage, Robbery, and Terrorism. We use the created dataset for the training and testing of the ConvLSTM (convolutional long short-term memory) neural network, which we designed. However, we also test the created dataset using other architectures. We use ConvLSTM architectures and 3D Resnet50, 3D Resnet101, and 3D Resnet152. With the created dataset and the architecture we designed, we obtained an accuracy of classification of 96.19% and a precision of 96.50%.

Review on Deep Learning Approaches for Anomaly Event Detection in Video Surveillance

“…Several researchers implemented two-stream CNN architectures for anomaly detection [43][44][45][46] and were shown to produce state-of-the-art results. [41] Simonyan et al [43] proposed a two-stream CNN, also known as a dual-stream CNN, to capture the spatial and temporal information, respectively. This model contains two networks (as shown in Figure 1) to capture the space and time information of video [5].…”

“…ConvLSTM is CNN combined with an LSTM network. It is like LSTM, but convolutional operations are done during layer transitions [41]. ConvLSTM performs on time-dependent data like video.…”

“…Few researchers divide their dataset into three sections: training, testing, and validation. [9,41] divided the dataset into 70% training, 20% testing, and 10% validation. Similarly, in [40], the dataset is split into 80% training, 10% validation, and 10% testing.…”

“…Refs. [9,41] divided the dataset into 70% training, 20% testing, and 10% validation. Similarly, in ref.…”

FRMDB: Face Recognition Using Multiple Points of View

“…The law enforcement sector is one such domain, leveraging AI and DL to serve crime investigations [ 2 ] by implementing applications increasingly able to autonomously detect suspicious activities [ 3 ]. For example, in applications such as violence detection [ 4 , 5 ], weapon detection [ 6 , 7 ], traffic accident detection [ 8 ], and human trajectory prediction [ 9 ], DL-based techniques exploit the availability of video surveillance systems, providing accurate and rich information to achieve security [ 10 ].…”