Deep Multi-Scale Features Fusion for Effective Violence Detection and Control Charts Visualization

Abstract

The study of automated video surveillance systems study using computer vision techniques is a hot research topic and has been deployed in many real-world CCTV environments. The main focus of the current systems is higher accuracy, while the assistance of surveillance experts in effective data analysis and instant decision making using efficient computer vision algorithms need researchers’ attentions. In this research, to the best of our knowledge, we are the first to introduce a process control technique: control charts for surveillance video data analysis. The control charts concept is merged with a novel deep learning-based violence detection framework. Different from the existing methods, the proposed technique considers the importance of spatial information, as well as temporal representations of the input video data, to detect human violence. The spatial information are fused with the temporal dimension of the deep learning model using a multi-scale strategy to ensure that the temporal information are properly assisted by the spatial representations at multi-levels. The proposed frameworks’ results are kept in the history-maintaining module of the control charts to validate the level of risks involved in the live input surveillance video. The detailed experimental results over the existing datasets and the real-world video data demonstrate that the proposed approach is a prominent solution towards automated surveillance with the pre- and post-analyses of violent events.

Literature Review of Deep-Learning-Based Detection of Violence in Video

“…Thus, it has been possible to observe the most used algorithms in phases 1 and 2 of the process of detecting violence in videos, as well as the most used combinations. [29,35,39,43,45,49,50,53,[56][57][58][59][60][62][63][64]66,71,74,76,77,[86][87][88]92].…”

“…Mumtaz et al [ 58 ] used VGG-19 (pre-trained CNN) whose output they fed into an LSTM. At the time of publication of the paper, and to the best of the authors’ knowledge, they were the first to introduce a process control technique: control charts for surveillance video data analysis.…”

“…Colors: Yellow corresponds to the use of CNNs, purple to skeleton-based algorithms, orange to manual techniques, red to those utilizing audio, gray to those employing transformers, and green to LSTM. Citations in descending order of accuracy: [ 29 , 35 , 39 , 43 , 45 , 49 , 50 , 53 , 56 , 57 , 58 , 59 , 60 , 62 , 63 , 64 , 66 , 71 , 74 , 76 , 77 , 86 , 87 , 88 , 92 ].…”