Abnormal Activity Recognition with Residual Attention-based ConvLSTM Architecture for Video Surveillance

Abstract

Human activity recognition (HAR) has become a highly researched area with numerous practical applications in public safety. Deep learning has revolutionized HAR by introducing novel approaches to tackle its challenges. Abnormal activity recognition enables prompt intervention and enhances public safety. Presently visionbased activity recognition techniques mainly use recurrent neural network (RNN) architectures like LSTM to handle sequential data dependency. However, this approach struggles to capture the spatial information between consecutive frames in video data, limiting their ability to learn spatiotemporal patterns. To address this issue, we introduce a layer called ResAttenConvLSTM2D, a variant of the ConvLSTM layer, and propose a novel architecture for solving the abnormal activity recognition problem. In the residual attention model, attention is applied to the residual connections, enabling the network to concentrate on portions of the input by calculating the attention score at each time iteration during model training. In addition, the proposed approach addresses the challenge of limited resources in handling video data by employing robust key frame extraction methods using an unsupervised K-Means algorithm. The proposed architecture is tested for benchmark datasets, i.e., AIRT Lab, hockey fight, and abnormal human activity with a classification accuracy of 90%, 96%, and 99% respectively, showing comparable accuracy or complexity compared to the state-of-the-art (SOTA) approaches.