Using eye-tracking to support interaction with layered 3D interfaces on stereoscopic displays

Abstract

In this paper, we investigate the concept of gaze-based interaction with 3D user interfaces. We currently see stereo vision displays becoming ubiquitous, particularly as autostereoscopy enables the perception of 3D content without the use of glasses. As a result, application areas for 3D beyond entertainment in cinema or at home emerge, including work settings, mobile phones, public displays, and cars. At the same time, eye tracking is hitting the consumer market with low-cost devices. We envision eye trackers in the future to be integrated with consumer devices (laptops, mobile phones, displays), hence allowing the user's gaze to be analyzed and used as input for interactive applications. A particular challenge when applying this concept to 3D displays is that current eye trackers provide the gaze point in 2D only (x and y coordinates). In this paper, we compare the performance of two methods that use the eye's physiology for calculating the gaze point in 3D space, hence enabling gaze-based interaction with stereoscopic content. Furthermore, we provide a comparison of gaze interaction in 2D and 3D with regard to user experience and performance. Our results show that with current technology, eye tracking on stereoscopic displays is possible with similar performance as on standard 2D screens.

Resolving Target Ambiguity in 3D Gaze Interaction through VOR Depth Estimation

“…Vergence. These techniques obtain the 3D gaze point via triangulation using either horizontal disparity between the left and the right 2D gaze points [1,7,9,11,33] or the inter-pupillary distance [2,15,22,25]. Others have also used machine learning techniques to estimate gaze depth from vergence [32,42].…”

Monocular gaze depth estimation using the vestibulo-ocular reflex

“…Techniques that directly calculate the vergence can estimate the 3D gaze point by intersecting multiple gaze rays from the left and the right eyes [Duchowski et al 2001;Hennessey* and Lawrence 2009]. Alternatively, vergence can be calculated indirectly, such as techniques that obtain the 3D gaze point via triangulation using either horizontal disparity between the left and the right 2D gaze points [Alt et al 2014a;Daugherty et al 2010;Duchowski et al , 2011Pfeiffer et al 2008] or the inter-pupillary distance [Alt et al 2014b;Gutierrez Mlot et al 2016;Ki and Kwon 2008;Kwon et al 2006]. Others have used machine learning techniques to estimate gaze depth from vergence [Orlosky et al 2016;Wang et al 2014].…”

“…The accommodation of the eyes -the process of changing the curvature of the lens to control optical power -can be measured using autorefractors to infer the gaze depth [Mercier et al 2017]. Another example is the work by Alt et al [Alt et al 2014a], which used pupil diameter to infer the depth of the gazed target when interaction with stereoscopic content. This technique is based on the assumption that the pupil diameter changes as a function of accommodation given that lighting conditions remain constant [Stephan Reichelt 2010].…”

Attention Engagement and Cognitive State Analysis for Augmented Reality Text Display Functions

“…Using an optical see-through display, a user can interact with the real environment as well as virtual information overlay [13,1]. Also, by measuring a user's eye gaze vectors in space, we can infer which plane he or she is interacting with out of multiple virtual planes [25,2].…”