3D Gaze Estimation for Head-Mounted Eye Tracking System With Auto-Calibration Method

Li, Meng; Li, Youfu; Liu, Hai

doi:10.1109/access.2020.2999633

Cited by 27 publications

(17 citation statements)

References 32 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, an average accuracy of 2.68° was achieved. More recent work [ 15 ] used an RGBD front camera and saliency-based methods to auto-calibrate a head-mounted binocular eye tracking system. They achieved an accuracy of 3.7° indoors and 4.0outdoors.…”

Section: Introductionmentioning

confidence: 99%

3D Gaze Estimation Using RGB-IR Cameras

Mokatren

Kuflik

Shimshoni

2022

Sensors

View full text Add to dashboard Cite

In this paper, we present a framework for 3D gaze estimation intended to identify the user’s focus of attention in a corneal imaging system. The framework uses a headset that consists of three cameras, a scene camera and two eye cameras: an IR camera and an RGB camera. The IR camera is used to continuously and reliably track the pupil and the RGB camera is used to acquire corneal images of the same eye. Deep learning algorithms are trained to detect the pupil in IR and RGB images and to compute a per user 3D model of the eye in real time. Once the 3D model is built, the 3D gaze direction is computed starting from the eyeball center and passing through the pupil center to the outside world. This model can also be used to transform the pupil position detected in the IR image into its corresponding position in the RGB image and to detect the gaze direction in the corneal image. This technique circumvents the problem of pupil detection in RGB images, which is especially difficult and unreliable when the scene is reflected in the corneal images. In our approach, the auto-calibration process is transparent and unobtrusive. Users do not have to be instructed to look at specific objects to calibrate the eye tracker. They need only to act and gaze normally. The framework was evaluated in a user study in realistic settings and the results are promising. It achieved a very low 3D gaze error (2.12) and very high accuracy in acquiring corneal images (intersection over union—IoU = 0.71). The framework may be used in a variety of real-world mobile scenarios (indoors, indoors near windows and outdoors) with high accuracy.

show abstract

Section: Introductionmentioning

confidence: 99%

3D Gaze Estimation Using RGB-IR Cameras

Mokatren

Kuflik

Shimshoni

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…The eyeball can generally be regarded as two intersecting spheres with deformations and the center and radius of the eyeball as well as the angular offset between visual and optical axes are determined during user calibration procedures. Further, approaches also proposed to use a neural network to fit an eyeball to an eye image [2], [24], [29], [30].…”

Section: B Eye Tracking Technologymentioning

confidence: 99%

An Eye Tracking based Aircraft Helmet Mounted Display Aiming System

Li¹,

Qiu²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

<p>In this paper, we comprehensively present new advancements towards the working-principle, algorithm, prototype setup and experimental characterization and feasibility demonstration of a new aircraft helmet mounted display aiming system. More specifically, we describe detailed design methodology and algorithm of an eye tracking based HMD aiming system. We also present a proof-of-concept prototype and conduct experiment tests in lab environment to characterize the performance of the proposed system. Lastly, through a comparison with related works, we demonstrate the advantages of the proposed HMD system for free head movement and no additional light source required.</p>

show abstract

“…There has been many gaze estimation studies where the camera has access to the eyes or face of a person whose gaze is predicted [25,31,35,46,49]. Model-based methods often use a geometric prior information, e.g., pupil center [19] or iris contours [36] to create an eye model to predict the gaze.…”

Section: Related Workmentioning

confidence: 99%

“…There is many evidence in the literature showing that eye gaze [10,25,34] (as well as its approximation in terms of head pose [7,8,11,26,33,53]) is an important cue, e.g., for the detection of social interactions. However, robust estimation of the eye gaze in unconstrained scenarios is only possible by using relatively expensive equipment (e.g., eye tracker [15,[42][43][44]) or by equipping a constrained area with cameras and other sensors, thus limiting the activity extent of the subject [25,31,35,46,49].…”

Section: Introductionmentioning

confidence: 99%

Predicting Gaze from Egocentric Social Interaction Videos and IMU Data

Thakur

Beyan

Morerio

et al. 2021

Proceedings of the 2021 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Gaze prediction in egocentric videos is a fairly new research topic, which might have several applications for assistive technology (e.g., supporting blind people in their daily interactions), security (e.g., attention tracking in risky work environments), education (e.g., augmented / mixed reality training simulators, immersive games) and so forth. Egocentric gaze is typically estimated from video while few works attempt to use inertial measurement unit (IMU) data, a sensor modality often available in wearable devices (e.g., augmented reality headsets). Instead, in this paper, we examine whether joint learning of egocentric video and corresponding IMU data can improve the first-person gaze prediction compared to using these modalities separately. In this respect, we propose a multimodal network and evaluate it on several unconstrained social interaction scenarios captured by a first-person perspective. The proposed multimodal network achieves better results compared to unimodal methods as well as several (multimodal) baselines, showing that using egocentric video together with IMU data can boost the first-person gaze estimation performance.

show abstract

3D Gaze Estimation for Head-Mounted Eye Tracking System With Auto-Calibration Method

Cited by 27 publications

References 32 publications

3D Gaze Estimation Using RGB-IR Cameras

3D Gaze Estimation Using RGB-IR Cameras

An Eye Tracking based Aircraft Helmet Mounted Display Aiming System

Predicting Gaze from Egocentric Social Interaction Videos and IMU Data

Contact Info

Product

Resources

About