Model-free non-rigid head pose tracking by joint shape and pose estimation

Abstract

Head pose estimation under non-rigid face movement is particularly useful in applications relating to eye-gaze tracking in less constrained scenarios, where the user is allowed to move naturally during tracking. Existing visionbased head pose estimation methods often require accurate initialisation and tracking of specific facial landmarks, while methods that handle non-rigid face deformations typically necessitate a preliminary training phase prior to head pose estimation. In this paper, we propose a method to estimate the head pose in real-time from the trajectories of a set of feature points spread randomly over the face region, without requiring a training phase or model-fitting of specific facial features. Conversely, our method exploits the 3-dimensional shape of the surface of interest, recovered via shape and motion factorisation, in combination with Kalman and particle filtering to determine the contribution of each feature point to the estimation of head pose based on a variance measure. Quantitative and qualitative results reveal the capability of our method in handling non-rigid face movement without deterioration of the head pose estimation accuracy.

Unobtrusive and pervasive video-based eye-gaze tracking

“…• Robustness to non-rigid face deformations: Non-rigid face deformations due to changes in facial expression are a common occurrence during tracking, yet this challenge is not typically addressed by the eyegaze tracking community. Head pose invariant methods that infer the head orientation from the relative configuration of specific facial features are particularly susceptible to non-rigid face deformations [158], since these often regard the face as a rigid body. In light of the challenges of pervasive eye-gaze tracking, a future direction is to develop methods that are robust to non-rigid face deformations, hence lifting further constraints on the natural user movement.…”