It’s Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation

Abstract

Eye gaze is an important non-verbal cue for human affect analysis. Recent gaze estimation work indicated that information from the full face region can benefit performance. Pushing this idea further, we propose an appearance-based method that, in contrast to a long-standing line of work in computer vision, only takes the full face image as input. Our method encodes the face image using a convolutional neural network with spatial weights applied on the feature maps to flexibly suppress or enhance information in different facial regions. Through extensive evaluation, we show that our full-face method significantly outperforms the state of the art for both 2D and 3D gaze estimation, achieving improvements of up to 14.3% on MPIIGaze and 27.7% on EYEDIAP for person-independent 3D gaze estimation. We further show that this improvement is consistent across different illumination conditions and gaze directions and particularly pronounced for the most challenging extreme head poses.

Gaze360: Physically Unconstrained Gaze Estimation in the Wild

“…Dataset statistics. Joint distributions of the gaze yaw and pitch for TabletGaze [10], MPIIFaceGaze [31], iTracker [12] and our Gaze360 dataset. The Mollweide projection used to visualize the full unit sphere surface.…”

“…The gaze prediction output regresses the angle of the gaze relative to the camera view. In previous work, 3D gaze was predicted as a unit gaze vector [17,34] or as its spherical coordinates [23,31]. We use spherical coordinates which we believe to be more naturally interpretable in this context.…”

Unsupervised Learning of Eye Gaze Representation from the Web

“…We have taken entire face as input instead of only eyes. According to [42], gaze can be more accurately predicted when the entire face is considered. Our proposed network contains five Convolution layers.…”

How Far Are We From Quantifying Visual Attention in Mobile HCI?

“…Most of these studies required special-purpose eye tracking equipment [2] that constrained users' mobility or manual data annotation [1] that prevented the study of natural attentive behaviour at scale. One way to overcome these limitations is to instead rely on the high-resolution front-facing cameras readily integrated into these devices in combination with computer vision for gaze estimation [5], [6], [7], [8]. However, despite significant progress in recent years, such methods are still too inaccurate to analyse attention in a fine-grained manner, e.g.…”