Joint face and head tracking inside multi-camera smart rooms

Abstract

The paper introduces a novel detection and tracking system that provides both frame-view and world-coordinate human location information, based on video from multiple synchronized and calibrated cameras with overlapping fields of view. The system is developed and evaluated for the specific scenario of a seminar lecturer presenting in front of an audience inside a "smart room", its aim being to track the lecturer's head centroid in the three-dimensional (3D) space and also yield two-dimensional (2D) face information in the available camera views. The proposed approach is primarily based on a statistical appearance model of human faces by means of wellknown AdaBoost-like face detectors, extended to address the head pose variation observed in the smart room scenario of interest. The appearance module is complemented by two novel components and assisted by a simple tracking drift detection mechanism. The first component of interest is the initialization module, which employs a spatio-temporal dynamic programming approach with appropriate penalty functions to obtain optimal 3D location hypotheses. The second is an adaptive subspace learning based 2D tracking scheme with a novel forgetting mechanism, introduced as a means to reduce tracking drift and increase robustness to illumination and head pose variation. System performance is benchmarked on an extensive database of realistic human interaction in the lecture smart room scenario, collected as part of the European integrated project "CHIL". The system consistently achieves excellent tracking precision, with a 3D mean tracking error of less than 16 cm, and is demonstrated to outperform four alternative tracking schemes. Furthermore, the proposed system performs relatively well in detecting frontal and near-frontal faces in the available frame views.

Fast lead star detection in entertainment videos

“…Researchers have explored various video specific applications for lead stars detection -anchor detection in news video [9], lead casts in comedy sitcoms [2], summarizing meetings [8], guest host detection [6,5] and locating the lecturer in smart rooms by tracking the face and head [10]. [3] uses affine invariant clustering to detect cast listing from movie.…”

Tracking identities and attention in smart environments - contributions and progress in the CHIL project

“…Second, a data-driven approach where 2D trackers operate independently on the separate camera views and the 2D tracks belonging to a same target are collected into a 3D track [28,52,56].…”

Camera-based eye blinks pattern detection for intelligent mouse

“…Motion analysis is another method for face detection [7,8]. Template matching combined with different classifiers is the other way for face and eye detection [9][10][11][12][13].…”