Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge

Abstract

This paper addresses the problem of joint detection and recounting of abnormal events in videos. Recounting of abnormal events, i.e., explaining why they are judged to be abnormal, is an unexplored but critical task in video surveillance, because it helps human observers quickly judge if they are false alarms or not. To describe the events in the human-understandable form for event recounting, learning generic knowledge about visual concepts (e.g., object and action) is crucial. Although convolutional neural networks (CNNs) have achieved promising results in learning such concepts, it remains an open question as to how to effectively use CNNs for abnormal event detection, mainly due to the environment-dependent nature of the anomaly detection. In this paper, we tackle this problem by integrating a generic CNN model and environment-dependent anomaly detectors. Our approach first learns CNN with multiple visual tasks to exploit semantic information that is useful for detecting and recounting abnormal events. By appropriately plugging the model into anomaly detectors, we can detect and recount abnormal events while taking advantage of the discriminative power of CNNs. Our approach outperforms the stateof-the-art on Avenue and UCSD Ped2 benchmarks for abnormal event detection and also produces promising results of abnormal event recounting. 1

SiTGRU: Single-Tunnelled Gated Recurrent Unit for Abnormality Detection

“…Hinami et al [17] proposed an approach that jointly detects and recounts abnormal events by exploiting generic knowledge. Multi-task Fast R-CNN is used to detect distinct visual concepts, which are then supplied to three separate anomaly detectors (viz., One-Class Support Vector Machine, Nearest Neighbour and Kernel Density Estimation) to measure anomaly scores of each visual concept in a scene and identify whether it is anomalous or not.…”

“…Specifically, we choose the Avenue dataset for this task as it has videos that contain only motion abnormalities in its test set. Following the approach presented by Hinami et al [17], we discard five videos which contain static appearance anomalies (i.e., video#1, video#2, video#8, video#9, video#10) from the test set and keep the rest sixteen videos. We call this test set Avenue16 and evaluate our trained model with it.…”

“…Abnormal Event Detection (AED) is a computer vision task for detecting abnormal events or behaviours from a given image or video. Automatic abnormal event detection has recently gained remarkable attention in the field of computer vision [12,17], in which humans are freed from controlling surveillance whenever an alert pops up. The surveillance system only focuses on abnormal events while the captured videos contain immense invaluable information.…”

“…In recent years, many abnormal event detection methods are proposed and have achieved great successes [12,17,34]. However, it is still a challenging problem due to two key factors.…”

“…(ii) The contentious definition of the term "abnormal" or "anomaly" causes significantly different solutions in different context (environment) [12,28,29]. Abnormality is not clearly defined as it is context-specific, leaving a room for unintentional subjectivity in data annotation [12,17,28]. For instance, Sun et al [34] de-fined "anomaly" as an event or scene that is rarely manifested.…”