Campus Abnormal Behavior Recognition With Temporal Segment Transformers

Abstract

The intelligent campus surveillance system is beneficial to improve safety in school. Abnormal behavior recognition, a field of action recognition in computer vision, plays an essential role in intelligent surveillance systems. Computer vision has been actively applied to action recognition systems based on Convolutional Neural Networks (CNNs). However, capturing sufficient motion sequence features from videos remains a significant challenge in action recognition. This work explores the challenges of video-based abnormal behavior recognition on campus. In addition, a novel framework is established on long-range temporal video structure modeling and a global sparse uniform sampling strategy that divides a video into three segments of identical durations and uniformly samples each snippet. The proposed method incorporates a consensus of three temporal segment transformers (TST) that globally connects patches and computes self-attention with joint spatiotemporal factorization. The proposed model is developed on the newly created campus abnormal behavior recognition (CABR50) dataset, which contains 50 human abnormal action classes with an average of over 700 clips per class. Experiments show that it is feasible to implement abnormal behavior recognition on campus and that the proposed method is competitive with other peer video recognition in terms of Top-1 and Top-5 recognition accuracy. The results suggest that TST-L+ can improve campus abnormal behavior recognition, corresponding to Top-1 and Top-5 accuracy results of 83.57% and 97.16%, respectively.

Polygonized Silhouettes and Polygon Coding Based Feature Representation for Human Action Recognition

“…Every vector ⃗ v lk in set V m has magnitude and angle. The magnitudes are calculated by using Equation (3).…”

“…As the k-mer degree increases, the number of groups and size of the feature vector decrease; thus, the processing speed of the classification increases. For error-free grouping, the TeC value must be divided without the remainder by the k-mer; so that, TeC is set to 360 and different k-mer values were used (3,4,5). Length of the feature vectors changes depending on these combinations.…”

“…Human action recognition and classification are vital in many computer vision applications, including human-computer interaction, sports event analysis, hand and body gesture recognition, video indexing, and video surveillance. Researchers have been working on video analytics for years, but it is becoming increasingly difficult to automatically recognize human actions and events due to the growing richness of video content [1], [2], [3]. There are various technologies such as cameras, depth sensors, and wearable devices that can accurately gather information about people's activities.…”