Pose Normalization for Eye Gaze Estimation and Facial Attribute Description from Still Images

Abstract

Abstract. Our goal is to obtain an eye gaze estimation and a face description based on attributes (e.g. glasses, beard or thick lips) from still images. An attribute-based face description reflects human vocabulary and is therefore adequate as face description. Head pose and eye gaze play an important role in human interaction and are a key element to extract interaction information from still images. Pose variation is a major challenge when analyzing them. Most current approaches for facial image analysis are not explicitly pose-invariant. To obtain a poseinvariant representation, we have to account the three dimensional nature of a face. A 3D Morphable Model (3DMM) of faces is used to obtain a dense 3D reconstruction of the face in the image. This Analysis-bySynthesis approach provides model parameters which contain an explicit face description and a dense model to image correspondence. However, the fit is restricted to the model space and cannot explain all variations. Our model only contains straight gaze directions and lacks high detail textural features. To overcome this limitations, we use the obtained correspondence in a discriminative approach. The dense correspondence is used to extract a pose-normalized version of the input image. The warped image contains all information from the original image and preserves gaze and detailed textural information. On the pose-normalized representation we train a regression function to obtain gaze estimation and attribute description. We provide results for pose-invariant gaze estimation on still images on the UUlm Head Pose and Gaze Database and attribute description on the Multi-PIE database. To the best of our knowledge, this is the first pose-invariant approach to estimate gaze from unconstrained still images.

Unobtrusive and pervasive video-based eye-gaze tracking

“…The results in Table 1 suggest that free head movement tends to introduce an increase in the gaze estimation error, when compared with the results achieved by methods that consider a frontal and stationary head pose, or which compensate for small head movement alone. Several of the methods that have been tested under free head movement in Table 1 [106,107,110,134,135,146,147], indeed report some of the highest gaze estimation errors among all the state-of-the-art methods that have been considered. In particular, some of the highest gaze estimation errors have been reported by appearance-based methods that address Challenge B, which relates to the estimation of gaze from sparse, synthesised or person-independent training samples, and which aim to compensate for free head movement as well [106,107,110].…”

“…One of the challenging aspects relating to the estimation of eye-gaze under different head orientations is the introduction of significant changes in eye image appearance that arise with a changing head pose. A subset of the methods that seek to address this problem propose to compensate for such changes in appearance by normalising the eye images to a frontal view based on an inverse rigid transformation of pre-computed head pose parameters [134][135][136][137][138]. The gaze direction, based on the eyeball rotation alone, is then estimated from the pose-normalised image and subsequently transformed by the pre-computed head pose parameters to compute a combined gaze estimate.…”

“…Methods that exploit the photometric appearance of the eye and necessitate the collection of training data for gaze estimation also benefit from pose-normalisation since this permits the collection of training samples under a frontal head pose alone [134,135,140], hence alleviating the need for a large corpus of training data that captures the eye appearance under different head orientations [38,40,44]. In this regard, the methods of [134,135,140] first estimate the head pose parameters by fitting a 3-dimensional morphable model to specific facial landmarks [135,140] or depth information acquired by a Kinect sensor [134]. By compensating for changes in head pose, the method proposed by Egger et al [135] permits the use of a single random forest regressor trained on frontal samples in order to estimate the gaze information from pose-normalised eye images.…”

“…In this regard, the methods of [134,135,140] first estimate the head pose parameters by fitting a 3-dimensional morphable model to specific facial landmarks [135,140] or depth information acquired by a Kinect sensor [134]. By compensating for changes in head pose, the method proposed by Egger et al [135] permits the use of a single random forest regressor trained on frontal samples in order to estimate the gaze information from pose-normalised eye images. Similarly, Mora and Odobez [134] train an adaptive linear regression (ALR) algorithm on pairs of frontal eye images and corresponding gaze information, as previously described in Section 5.2.1, permitting the estimation of gaze based on eyeball rotation within a frontal head reference frame.…”

“…Most of the aforementioned eye-gaze tracking methods employ a head pose estimation approach that necessitates the identification and tracking of specific facial landmarks [26,27,135,136,141,142]. This requires that the range of possible head rotation angles is constrained such that the entire set of facial landmarks remains visible during tracking, a condition which may limit the usability of these methods in wide spaces that often necessitate large head rotations.…”