InvisibleEye

Abstract

Analysis of everyday human gaze behaviour has significant potential for ubiquitous computing, as evidenced by a large body of work in gaze-based human-computer interaction, attentive user interfaces, and eye-based user modelling. However, current mobile eye trackers are still obtrusive, which not only makes them uncomfortable to wear and socially unacceptable in daily life, but also prevents them from being widely adopted in the social and behavioural sciences. To address these challenges we present InvisibleEye, a novel approach for mobile eye tracking that uses millimetre-size RGB cameras that can be fully embedded into normal glasses frames. To compensate for the cameras’ low image resolution of only a few pixels, our approach uses multiple cameras to capture different views of the eye, as well as learning-based gaze estimation to directly regress from eye images to gaze directions. We prototypically implement our system and characterise its performance on three large-scale, increasingly realistic, and thus challenging datasets: 1) eye images synthesised using a recent computer graphics eye region model, 2) real eye images recorded of 17 participants under controlled lighting, and 3) eye images recorded of four participants over the course of four recording sessions in a mobile setting. We show that InvisibleEye achieves a top person-specific gaze estimation accuracy of 1.79° using four cameras with a resolution of only 5 × 5 pixels. Our evaluations not only demonstrate the feasibility of this novel approach but, more importantly, underline its significant potential for finally realising the vision of invisible mobile eye tracking and pervasive attentive user interfaces.

Anticipating Averted Gaze in Dyadic Interactions

“…Analysing social gaze in conversations either requires specialised mobile eye tracking equipment [Kassner et al 2014;Tonsen et al 2017] or computational methods for gaze estimation and eye contact detection from off-the-shelf RGB cameras -the latter research area in computer vision has received particular attention in recent years. Gaze estimation methods can be roughly divided in model-based and appearance-based [Hansen and Ji 2009]: While model-based approaches use a geometric model of the human eye to perform gaze estimation [Valenti et al 2011;Wood et al 2015;Yamazoe et al 2008], appearance based methods directly regress the gaze from the image input [Lu et al 2012;Zhang et al 2019].…”

Forecasting user attention during everyday mobile interactions using device-integrated and wearable sensors

“…Fourth, the current hardware setup is rather bulky (head-mounted mobile eye tracker, multiple cameras, mobile phone, laptop backpack), which might have influenced participants' attentive behaviour. Therefore, investigating in-the-wild studies with participants' awareness about the recording will be an interesting future project [29,37] Fully integrating the required cameras is an important direction for future work, but likely to be feasible given recent advances in fully embedded head-mounted eye tracking [43].…”

Optimization-Based Eye Tracking using Deflectometric Information

“…Image-based methods retrieve geometrical information about the gaze direction from 2D features in captured 2D images of the eye [1,18,37,41]. Pupil, iris, limbus, or eyelids, are popular candidates for 2D features to calculate eye position and gaze direction.…”