Guiding Visual Surveillance by Tracking Human Attention

Abstract

We describe a novel method for directing the attention of an automated surveillance system. Our starting premise is that the attention of people in a scene can be used as an indicator of interesting areas and events. To determine people's attention from passive visual observations we develop a system for automatic tracking and detection of individual heads to infer their gaze direction. The former is achieved by combining a histogram of oriented gradient (HOG) based head detector with frame-to-frame tracking using multiple point features to provide stable head images. The latter is achieved using a head pose classification method which uses randomised ferns with decision branches based on both HOG and colour based features to determine a coarse gaze direction for each person in the scene. By building both static and temporally varying maps of areas where people look we are able to identify interesting regions.

Security and Surveillance

“…corresponding facial landmarks such as eyes and lips to a set of trained poses. Recent studies have attempted to estimate head pose in low-resolution images [8] as well as crowded surveillance videos [52]. In addition to head pose, body posture configuration [46] and gait [49] may also play an important role in human intent inference.…”

“…Alert is generated for an immediate review by security personnel if multiple persons enter a restricted area while only one of them is authorised by the access control system, e.g. Mate video analytics 8 . 4 A set of real-world datasets and alarm definitions are released as the Image Library for Intelligent Detection Systems (i-LIDS), a UK government Home Office Scientific Development Branch (HOSDB) benchmark for video analytics systems [63], which has also been adopted by the US National Institute of Standards and Technology (NIST).…”

Human behavior analysis in video surveillance: A Social Signal Processing perspective

“…For example, in [65], and, independently, in [66] the idea was to infer what part of the scene is seen more frequently by people, thus creating a sort of interest maps. This may serve to highlight individuals that are focused on particular portions of the environment for a long time: if the observed target is critical (for example, an ATM machine) a threatening behavior could be inferred (PROBLEM 1).…”

Improving Multi-frame Data Association with Sparse Representations for Robust Near-online Multi-object Tracking

“…These benchmarks are composed of training and testing sets [36][37][38][39][40][41] with public detections, given by Aggregate Channel Features (ACF) pedestrian detector [42] in the case of the 2DMOT2015 and a Deformable Part Model (DPM) [43] for the MOT16. The metrics employed by these benchmarks are based on the widely accepted CLEARMOT metrics [44].…”