Gaze‐driven Object Tracking for Real Time Rendering

Abstract

To efficiently deploy eye-tracking within 3D graphics applications, we present a new probabilistic method that predicts the patterns of user's eye fixations in animated 3D scenes from noisy eye-tracker data. The proposed method utilises both the eye-tracker data and the known information about the 3D scene to improve the accuracy, robustness and stability. Eye-tracking can thus be used, for example, to induce focal cues via gaze-contingent depth-of-field rendering, add intuitive controls to a video game, and create a highly reliable scene-aware saliency model. The computed probabilities rely on the consistency of the gaze scan-paths to the position and velocity of a moving or stationary target. The temporal characteristic of eye fixations is imposed by a Hidden Markov model, which steers the solution towards the most probable fixation patterns. The derivation of the algorithm is driven by the data from two eye-tracking experiments: the first experiment provides actual eye-tracker readings and the position of the target to be tracked. The second experiment is used to derive a JND-scaled (Just Noticeable Difference) quality metric that quantifies the perceived loss of quality due to the errors of the tracking algorithm. Data from both experiments are used to justify design choices, and to calibrate and validate the tracking algorithms. This novel method outperforms commonly used fixation algorithms and is able to track objects smaller then the nominal error of an eye-tracker.

The effects of fast disparity adjustment in gaze-controlled stereoscopic applications

“…In addition to that, we handed out a rating scale where participants cast a subjective judgment on stereo comfort and stereo depth. Hillaire et al [2008], and very recently Mantiuk et al [2013], proposed to use an eye-tracker to control DOF simulations in interactive applications. Eye tracking was also used in applications attempting to reduce stereo viewing discomfort by DOFblurring [Talmi and Liu 1999;Blum et al 2010;Leroy et al 2012].…”

Gaze-to-object mapping during visual search in 3D virtual environments

“…This could be done, for instance, with Hidden Markov Chains (cf. [Mantiuk et al 2013]). However, the choice of the temporal processing method is also a matter of the application where GTOM is used.…”

“…A straightforward, but effective strategy, is to exploit the strong tendency of users to focus at the center of an object [Henderson 1993]. Similar to [Xu et al 2008] (for mapping to words in a text documents) and [Mantiuk et al 2013](for mapping to small objects), we place a monovariate, but two-dimensional, Gaussian at the center of gravity of the object with a kernel size θ s :…”

“…Note also that Ray Casting and Center Of Gravity Mapping could not immediately be used in our framework and had to be adjusted for the use with an item buffer. Mantiuk et al [Mantiuk et al 2013] has introduced the idea of using motion information together with Gaussian distance metrics. The other three methods (i.e., Closest Feature Mapping, normalized Closest Feature Mapping/Fovea Splatting) are, to the best of our knowledge, first proposed in this work.…”

Gaze-to-Object Mapping during Visual Search in 3D Virtual Environments

“…This strategy yields an importance value which can be used as a probability prediction. An alternative to rendering an item buffer, proposed recently by Mantiuk et al [2013], is to sparsely attach target points to object surfaces and use those to compute the distance between gaze and these target points. They further proposed to utilize motion information to increase accuracy, since there is a certain amount of correlation between gaze movement trajectories and the motion of an attended target point, which can be transformed to a probability estimate.…”