Anomaly3D: Video anomaly detection based on 3D-normality clusters

Auto-encoder has been widely used in video anomaly detection which aims to detect abnormal segments in video surveillance. However, the previous auto-encoder methods preferred to reconstruct a model of the normal event that only trains on normal samples due to the lack of abnormal samples, which may lead to self-reconstruction and cannot guarantee a larger reconstruction error for an abnormal event. In this paper, a novel Two-Stream spatial-temporal auto-encoder (Two-Stream STAE) network with adversarial training is designed for video anomaly detection. The Two-Stream STAE network is composed of a spatial auto-encoder stream and a temporal auto-encoder stream. The spatial stream reconstructed the appearance model of the normal event by extracting features of detected objects, and the temporal stream encoded the stacked optical flow maps to reconstruct the temporal model of the normal event. In order to enlarge the reconstruction errors of the abnormal events in the network, an adversarial training branch that takes abnormal samples as input is proposed to train the model. A pseudo-abnormal dataset is also built to solve the problem of the lack of abnormal samples. Experimental results on three public benchmark datasets demonstrate the strong competitiveness of our proposed method when comparing with the state-of-the-art methods.INDEX TERMS Video anomaly detection, spatial-temporal auto-encoder, two-stream network , adversarial training.

show abstract

“…al [20] 88.3 18.9 83.9 -Ramachandra et. al [21] 94.0 14.1 93.0 -Anomaly3D [22] 95.8 ---Hao et. al [31] 96.9…”

Section: Methodsmentioning

confidence: 99%

Two-Stream Spatial-Temporal Auto-Encoder With Adversarial Training for Video Anomaly Detection

Guo

Liu

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The target segmentation of neuromorphic vision is a technique of dividing the pulse flow into several specific and characteristic regions and extracting the precise location of the target of interest. Asad et al (Asad et al, 2021) performed spatiotemporal clustering of the pulse flow and constructed a stereo vision system, which can perform real-time segmentation and behavior analysis for multiple pedestrians. Chen et al (Chen et al, 2018) proposed a real-time clustering tracking algorithm, which can segment and track vehicle objects in traffic scenes in real time.…”

Section: Object Segmentationmentioning

confidence: 99%

Artificial intelligence-based spatio-temporal vision sensors: applications and prospects

Li,

Sun

2023

Front. Mater.

View full text Add to dashboard Cite

Introduction: Advances in machine vision and mobile electronics will be accelerated by the creation of sophisticated optoelectronic vision sensors that allow for sophisticated picture recognition of visual information and data pre-processing. Several new types of vision sensors have been devised in the last decade to solve these drawbacks, one of which is neuromorphic vision sensors, which have exciting qualities such as high temporal resolution, broad dynamic range, and low energy consumption. Neuromorphic sensors are inspired by the working principles of biological sensory neurons and would be useful in telemedicine, health surveillance, security monitoring, automatic driving, intelligent robots, and other applications of the Internet of Things.Methods: This paper provides a comprehensive review of various state-of-the-art AI vision sensors and frameworks.Results: The fundamental signal processing techniques deployed and the associated challenges were discussed.Discussion: Finally, the role of vision sensors in computer vision is also discussed.

show abstract

“…Reconstruction based VAD approaches detect anomalies by reconstruction of video frames, where low and high reconstruction errors represent normality and abnormality, respectively. Reconstruction based deep approaches largely include those based on autoencoders (AE) [2,5] and its variants such as convolutional autoencoders (CAE) [6][7][8][9]. Prediction based deep approaches detect anomalies by predicting current frame features using that of previous frames.…”

Section: Introductionmentioning

confidence: 99%

Transformer Based Self-Context Aware Prediction for Few-Shot Anomaly Detection in Videos

Pillai

Verma

Sen

2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Anomaly detection in videos is a challenging task as anomalies in different videos are of different kinds. Therefore, a promising way to approach video anomaly detection is by learning the non-anomalous nature of the video at hand. To this end, we propose a one-class few-shot learning driven transformer based approach for anomaly detection in videos that is self-context aware. Features from the first few consecutive non-anomalous frames in a video are used to train the transformer in predicting the non-anomalous feature of the subsequent frame. This takes place under the attention of a self-context learned from the input features themselves. After the learning, given a few previous frames, the videospecific transformer is used to infer if a frame is anomalous or not by comparing the feature predicted by it with the actual. The effectiveness of the proposed method with respect to the state-of-the-art is demonstrated through qualitative and quantitative results on different standard datasets. We also study the positive effect of the self-context used in our approach.

show abstract

Anomaly3D: Video anomaly detection based on 3D-normality clusters

Cited by 20 publications

References 27 publications

Two-Stream Spatial-Temporal Auto-Encoder With Adversarial Training for Video Anomaly Detection

Two-Stream Spatial-Temporal Auto-Encoder With Adversarial Training for Video Anomaly Detection

Artificial intelligence-based spatio-temporal vision sensors: applications and prospects

Transformer Based Self-Context Aware Prediction for Few-Shot Anomaly Detection in Videos

Contact Info

Product

Resources

About