Attribute-based Representations for Accurate and Interpretable Video Anomaly Detection

Reiss, Tal; Hoshen, Yedid

doi:10.48550/arxiv.2212.00789

Cited by 2 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, their method is object-centric, so it cannot be applied to abnormal situations that a typical object detector cannot detect, such as explosion and arson in the UCFcrime dataset [ 11 ]. Another effective object-centric anomaly detection method by Reiss et al [ 10 ] shows even better results than Georgescu et al’s [ 9 ]. They calculate a feature from optical flow, 2D skeleton, and deep feature from CLIP [ 23 ] for detected objects in the training dataset.…”

Section: Related Workmentioning

confidence: 99%

“…In the inference, they estimate density using a Gaussian mixture model (GMM) for optical flow and k NN with the training dataset for 2D skeleton and deep feature. However, k NN distances from Reiss et al [ 10 ] cannot be used as an unsupervised approach, because an abnormal feature in training data makes k NN distance small, which makes the anomaly score for abnormal data become small. All of these OCC approaches require normal-only training dataset and, due to the absence of abnormal data, their performance is limited.…”

Section: Related Workmentioning

confidence: 99%

“…According to the dataset configuration, the approaches for anomaly detection in video surveillance can be categorized into one-class classification, weakly supervised, and unsupervised techniques, following the terms used in GCL [ 1 ]. The one-class classification approach uses a training dataset consisting of only normal situations [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 ]. In the weakly supervised approach, video-level labeled abnormal data are used for training, in addition to the normal data used in the one-class classification approach [ 11 , 12 , 13 , 14 , 15 , 16 , 17 ].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Unsupervised Video Anomaly Detection Based on Similarity with Predefined Text Descriptions

Kim

Yoon

Choi

et al. 2023

Sensors

View full text Add to dashboard Cite

Research on video anomaly detection has mainly been based on video data. However, many real-world cases involve users who can conceive potential normal and abnormal situations within the anomaly detection domain. This domain knowledge can be conveniently expressed as text descriptions, such as “walking” or “people fighting”, which can be easily obtained, customized for specific applications, and applied to unseen abnormal videos not included in the training dataset. We explore the potential of using these text descriptions with unlabeled video datasets. We use large language models to obtain text descriptions and leverage them to detect abnormal frames by calculating the cosine similarity between the input frame and text descriptions using the CLIP visual language model. To enhance the performance, we refined the CLIP-derived cosine similarity using an unlabeled dataset and the proposed text-conditional similarity, which is a similarity measure between two vectors based on additional learnable parameters and a triplet loss. The proposed method has a simple training and inference process that avoids the computationally intensive analyses of optical flow or multiple frames. The experimental results demonstrate that the proposed method outperforms unsupervised methods by showing 8% and 13% better AUC scores for the ShanghaiTech and UCFcrime datasets, respectively. Although the proposed method shows −6% and −5% than weakly supervised methods for those datasets, in abnormal videos, the proposed method shows 17% and 5% better AUC scores, which means that the proposed method shows comparable results with weakly supervised methods that require resource-intensive dataset labeling. These outcomes validate the potential of using text descriptions in unsupervised video anomaly detection.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%