Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning

Tian, Yu; Pang, Guansong; Chen, Yuanhong; Singh, Rajvinder; Verjans, Johan; Carneiro, Gustavo

doi:10.1109/iccv48922.2021.00493

Cited by 218 publications

(205 citation statements)

References 57 publications

Supporting

Mentioning

132

Contrasting

Order By: Relevance

“…Following [19,23], each video is divided into 32 video snippets, i.e., T = 32. For all experiments, we set k = 3 in (4).…”

Section: Implementation Detailsmentioning

confidence: 99%

“…The learning rate is set to 0.001. Following [19,23], each mini-batch consists of samples from 32 randomly selected normal and abnormal videos. The method is implemented in PyTorch [17] and trained with a NVIDIA 3090 GPU.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…However, UAD approaches do not use training images or snippets (i.e., a set of consecutive video frames) containing polyps, so they are ineffective in recognising polyps of diverse characteristics, especially those that are small, partially visible, or irregularly shaped. As shown in a number of recent studies [15,16,19,21,23,28], incorporating some knowledge about anomalies into the training of anomaly detectors has improved the detection accuracy of hard anomalies. For example, weakly-supervised video anomaly detection (WVAD) [19,23,28] relies on videolevel labelled data to train detection models.…”

Section: Introductionmentioning

confidence: 99%

“…As shown in a number of recent studies [15,16,19,21,23,28], incorporating some knowledge about anomalies into the training of anomaly detectors has improved the detection accuracy of hard anomalies. For example, weakly-supervised video anomaly detection (WVAD) [19,23,28] relies on videolevel labelled data to train detection models. The video-level labels only indicate whether the whole video contains anomalies or not, which is easier to acquire than fully-labelled datasets with frame-level annotations.…”

Section: Introductionmentioning

confidence: 99%

“…Most existing WVAD methods [7,19,23,28,30] rely on multiple instance learning (MIL), in which all snippets in a normal video are treated as normal snippets, while each abnormal video is assumed to have at least one abnormal snippet. This approach can utilise video-level labels to train an anomaly-informed detector to find anomalous frames, but MIL methods often fail to select rare abnormal snippets in anomalous videos, especially the challenging abnormal snippets that have subtle visual appearance differences from the normal ones (e.g., small and flat colon polyps or frames with partially visible polyps-see Fig.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Contrastive Transformer-based Multiple Instance Learning for Weakly Supervised Polyp Frame Detection

Tian¹,

Pang²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Current polyp detection methods from colonoscopy videos use exclusively normal (i.e., healthy) training images, which i) ignore the importance of temporal information in consecutive video frames, and ii) lack knowledge about the polyps. Consequently, they often have high detection errors, especially on challenging polyp cases (e.g., small, flat, or partially visible polyps). In this work, we formulate polyp detection as a weakly-supervised anomaly detection task that uses video-level labelled training data to detect frame-level polyps. In particular, we propose a novel convolutional transformer-based multiple instance learning method designed to identify abnormal frames (i.e., frames with polyps) from anomalous videos (i.e., videos containing at least one frame with polyp). In our method, local and global temporal dependencies are seamlessly captured while we simultaneously optimise video and snippet-level anomaly scores. A contrastive snippet mining method is also proposed to enable an effective modelling of the challenging polyp cases. The resulting method achieves a detection accuracy that is substantially better than current state-of-the-art approaches on a new large-scale colonoscopy video dataset introduced in this work. Our code and dataset will be publicly available upon acceptance.

show abstract

“…Following [19,23], each video is divided into 32 video snippets, i.e., T = 32. For all experiments, we set k = 3 in (4).…”

Section: Implementation Detailsmentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%