Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos

Morais, Romero; Le, Vuong; Tran, Truyen; Saha, Budhaditya; Mansour, Moussa Reda; Venkatesh, Svetha

doi:10.1109/cvpr.2019.01227

Cited by 258 publications

(218 citation statements)

References 26 publications

(42 reference statements)

Supporting

Mentioning

197

Contrasting

Order By: Relevance

“…However, prediction by a per-frame basis leads to a bias to background [20], and [48] proposes attention mechanism to ease the issue. Another natural idea is to combine prediction with reconstruction as a hybrid VAD solution: [46] design a spatio-temporal CAE (ST-CAE), in which an encoder is followed by two decoders for reconstruction and prediction purpose respectively; [28] reconstructs and predicts human skeletons by a message-passing encoder-decoder RNN (MPED-RNN); [42] integrates reconstruction into prediction by a predictive coding network based framework (AnoPCN); [36] conducts prediction and reconstruction in a sequential manner.…”

Section: Related Workmentioning

confidence: 99%

Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Events

Wang

Cai

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

156

101

View full text Add to dashboard Cite

As a vital topic in media content interpretation, video anomaly detection (VAD) has made fruitful progress via deep neural network (DNN). However, existing methods usually follow a reconstruction or frame prediction routine. They suffer from two gaps: (1) They cannot localize video activities in a both precise and comprehensive manner. (2) They lack sufficient abilities to utilize high-level semantics and temporal context information. Inspired by frequently-used cloze test in language study, we propose a brand-new VAD solution named Video Event Completion (VEC) to bridge gaps above: First, we propose a novel pipeline to achieve both precise and comprehensive enclosure of video activities. Appearance and motion are exploited as mutually complimentary cues to localize regions of interest (RoIs). A normalized spatio-temporal cube (STC) is built from each RoI as a video event, which lays the foundation of VEC and serves as a basic processing unit. Second, we encourage DNN to capture high-level semantics by solving a visual cloze test. To build such a visual cloze test, a certain patch of STC is erased to yield an incomplete event (IE). The DNN learns to restore the original video event from the IE by inferring the missing patch. Third, to incorporate richer motion dynamics, another DNN is trained to infer erased patches' optical flow. Finally, two ensemble strategies using different types of IE and modalities are proposed to boost VAD performance, so as to fully exploit the temporal context and modality information for VAD. VEC can consistently outperform state-ofthe-art methods by a notable margin (typically 1.5%-5% AUROC) on commonly-used VAD benchmarks. Our codes and results can be verified at github.com/ yuguangnudt/ VEC_VAD.

show abstract

Section: Related Workmentioning

confidence: 99%

Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Events

Wang

Cai

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

156

101

View full text Add to dashboard Cite

show abstract

“…These feature vectors are automatically classified using anomaly detection algorithms developed during pilot work. 49 50 Based on a k —nearest neighbour classification approach on 265 video recordings of babies, and a feature based on the histogram of the optical flow, the accuracy for automated GMA was 72.9%. 50…”

Section: Methodsmentioning

confidence: 99%

Early Moves: a protocol for a population-based prospective cohort study to establish general movements as an early biomarker of cognitive impairment in infants

et al. 2021

Self Cite

View full text Add to dashboard Cite

IntroductionThe current diagnostic pathways for cognitive impairment rarely identify babies at risk before 2 years of age. Very early detection and timely targeted intervention has potential to improve outcomes for these children and support them to reach their full life potential. Early Moves aims to identify early biomarkers, including general movements (GMs), for babies at risk of cognitive impairment, allowing early intervention within critical developmental windows to enable these children to have the best possible start to life.Method and analysisEarly Moves is a double-masked prospective cohort study that will recruit 3000 term and preterm babies from a secondary care setting. Early Moves will determine the diagnostic value of abnormal GMs (at writhing and fidgety age) for mild, moderate and severe cognitive delay at 2 years measured by the Bayley-4. Parents will use the Baby Moves smartphone application to video their babies’ GMs. Trained GMs assessors will be masked to any risk factors and assessors of the primary outcome will be masked to the GMs result. Automated scoring of GMs will be developed through applying machine-based learning to the data and the predictive value for an abnormal GM will be investigated. Screening algorithms for identification of children at risk of cognitive impairment, using the GM assessment (GMA), and routinely collected social and environmental profile data will be developed to allow more accurate prediction of cognitive outcome at 2 years. A cost evaluation for GMA implementation in preparation for national implementation will be undertaken including exploring the relationship between cognitive status and healthcare utilisation, medical costs, health-related quality of life and caregiver burden.Ethics and disseminationEthics approval has been granted by the Medical Research Ethics Committee of Joondalup Health Services and the Health Service Human Research Ethics Committee (1902) of Curtin University (HRE2019-0739).Trial registration numberACTRN12619001422112.

show abstract

“…Sultani et al [1] considered normal and anomalous videos as bags and video segments as instances in multiple instance learning (MIL) and used a 3D convolutional network to extract spatial-temporal information. Morais et al [15] learned the skeleton trajectories with a message-passing encoder-decoder recurrent network.…”

Section: Related Work a Anomaly Detectionmentioning

confidence: 99%

Dual Discriminator Generative Adversarial Network for Video Anomaly Detection

2020

View full text Add to dashboard Cite

Video anomaly detection is an essential task because of its numerous applications in various areas. Because of the rarity of abnormal events and the complicated characteristic of videos, video anomaly detection is challenging and has been studied for a long time. In this paper, we propose a semi-supervised approach with a dual discriminator-based generative adversarial network structure. Our method considers more motion information in video clips compared with previous approaches. Specifically, in the training phase, we predict future frames for normal events via a generator and attempt to force the predicted frames to be similar to their ground truths. In addition, we utilize both a frame discriminator and motion discriminator to adverse the generator to generate more realistic and consecutive frames. The frame discriminator attempts to determine whether the input frames are generated or original frames sampled from the normal video. The motion discriminator attempts to determine whether the given optical flows are real or fake. Fake optical flows are estimated from generated frames and adjacent frames, and real optical flows are estimated from the real frames sampled from original videos. Then, in the testing phase, we evaluate the quality of predicted frames to obtain the regular score, and we consider those frames with lower prediction qualities as abnormal frames. The results of experiments on three publicly available datasets demonstrate the effectiveness of our proposed method.

show abstract

Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos

Cited by 258 publications

References 26 publications

Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Events

Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Events

Early Moves: a protocol for a population-based prospective cohort study to establish general movements as an early biomarker of cognitive impairment in infants

Dual Discriminator Generative Adversarial Network for Video Anomaly Detection

Contact Info

Product

Resources

About