Smartphone eye tracking toolbox

Abstract

Human Interaction with mobile devices has recently been established as application field in eye tracking research. Current technologies for gaze recovery on mobile displays cannot enable fully natural interaction with the mobile device: users are conditioned to interact with tightly mounted displays or distracted by markers in their view. We propose a novel approach that captures point-of-regards (PORs) with eye tracking glasses (ETG) and then uses computer vision methodology for the robust localization of the smartphone in the head camera video. We present an integrated software package, i.e., the Smartphone Eye Tracking Toolbox (SMET) that enables accurate gaze recovery on mobile displays with heat mapping of recent attention. We report the performance of the computer vision approach and demonstrate it with various natural interaction scenarios using the SMET Toolbox, enable ROI settings on the mobile display and show results from eye movement analysis, such as, ROI dwell time and statistics on eye gaze event (saccades, fixations).

Imaging Time Series of Eye Tracking Data to Classify Attentional States

“…Our knowledge about saccades and fixations, their cause and reason, and their connection to the current mental state of the observed person has increased immensely since then and the practice of eye tracking has found many applications. In addition to the mentioned research interests, human gaze tracking is widely used in consumer and marketing research (Wedel and Pieters, 2008) or as an input mechanism for technical devices, such as smartphones (Paletta et al, 2014) and Augmented and Virtual Reality glasses (Miller, 2020). Some applications are mainly interested in the direction of the gaze (i.e., to predict salient regions of web pages as in Buscher et al, 2009).…”

The past, present, and future of gaze-enabled handheld mobile devices

“…The first works to explore eye tracking on handheld mobile devices overcame the hardware limitations in different ways. Some augmented the user by having participants wear a headmounted eye tracker [35,75,87], while others augmented the device either by building their own hardware [24,111,85] or by using remote commercial eye trackers [25,86].…”

Forecasting user attention during everyday mobile interactions using device-integrated and wearable sensors

“…Wood and Bulling used a model-based gaze estimation approach on an off-the-shelf tablet and achieved an average gaze estimation accuracy of 6.88°at 12 frames per second [52] while Vaitukaitis and Bulling combined methods from image processing, computer vision and pattern recognition to detect eye gestures using the built-in front-facing camera [48]. Jiang et al proposed a method to estimate visual attention on objects of interest in the user's environment by jointly exploiting the phone's front-and rear-facing cameras [19] while Paletta et al investigated accurate gaze estimation on mobile phones using a computer vision method to detect the phone in an eye tracker's scene video [33]. While all of these works focused on estimating gaze spatially on the device screen, we are the first to predict attention allocation temporally.…”