Gaze360: Physically Unconstrained Gaze Estimation in the Wild

Abstract

Understanding where people are looking is an informative social cue. In this work, we present Gaze360, a large-scale gaze-tracking dataset and method for robust 3D gaze estimation in unconstrained images. Our dataset consists of 238 subjects in indoor and outdoor environments with labelled 3D gaze across a wide range of head poses and distances. It is the largest publicly available dataset of its kind by both subject and variety, made possible by a simple and efficient collection method. Our proposed 3D gaze model extends existing models to include temporal information and to directly output an estimate of gaze uncertainty. We demonstrate the benefits of our model via an ablation study, and show its generalization performance via a cross-dataset evaluation against other recent gaze benchmark datasets. We furthermore propose a simple self-supervised approach to improve cross-dataset domain adaptation. Finally, we demonstrate an application of our model for estimating customer attention in a supermarket setting. Our dataset and models are available at

EyeShopper

“…We present experimental results with real CCTV video footage in different locations and with different quality resolutions. Gaze estimation results show that EyeShopper can achieve a mean absolute error of 19 • , very close to the performance of other state-of-the-art systems that rely on frontal face images such as Hopenet [38], while being more framerate agnostic than more sophisticate algorithms [18].…”

“…Recent advances in object recognition have demonstrated how computer vision methods can enable capabilities such as product recognition [11] or gaze estimation [18,50]. All the aforementioned proposals have one or more of the following weaknesses: (i) insufficient precision or accuracy, (ii) challenging deployment, in the case of techniques that need installed devices to function (e.g., in each Amazon Go store hundreds of cameras are installed), (iii) threat private visual boundaries, in the case of computer vision techniques as shoppers do not want to feel observed [8,30] and might not want their faces to be recorded for post-analytic purposes, or (iv) require attached devices to the shopper (wearables, smartphones).…”

“…In this work, we present EyeShopper, a framework for collection and analysis of CCTV camera feeds that predicts which products are being gazed by shoppers in an store. In contrast to several state-of-the-art models, such as Hopenet [38] and Gaze360 [18], that mostly work with frontal face images, EyeShopper can estimate shoppers' gaze with high accuracy using back head images. EyeShopper by having a mean absolute error of 19 • shows a 28% improvement over other head pose estimation algorithms for back head images and can be used in conjuction with frontal face gaze estimation systems to achieve the best of both worlds.…”

“…2) Synthetic dataset. Considering the nonavailability of a dataset composed of back head images for head pose estimation [18], we generate a synthetic one. The dataset contains 144,000 back head images of different illumination conditions and perspectives, simulating different camera positions.…”

Accurate Regression-Based 3D Gaze Estimation Using Multiple Mapping Surfaces

“…Regression functions are calibrated by simultaneously capturing eye features and calibration points on a calibration plane. Similar to these methods, appearance-based methods use eye appearance [18], [19] as eye features. They leverage large training data with convolutional neural networks to achieve unconstrained gaze estimation.…”