Generalizing Eye Tracking With Bayesian Adversarial Learning

Wang, Kang; Zhao, Rui; Su, Hui; Ji, Qiang

doi:10.1109/cvpr.2019.01218

Cited by 86 publications

(61 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ultimate solution is to collect training data that covers the whole data space, however, it is practically impossible. Several studies have attempted to extract subjectinvariant features from eye images [38], [51], [40]. Park et al [38] convert the original eye images into a unified gaze representation, which is a pictorial representation of the eyeball, the iris and the pupil.…”

Section: A Deep Feature From Appearancementioning

confidence: 99%

Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark

Cheng,

Wang,

Bao

et al. 2021

Preprint

View full text Add to dashboard Cite

Gaze estimation reveals where a person is looking. It is an important clue for understanding human intention. The recent development of deep learning has revolutionized many computer vision tasks, the appearance-based gaze estimation is no exception. However, it lacks a guideline for designing deep learning algorithms for gaze estimation tasks. In this paper, we present a comprehensive review of the appearance-based gaze estimation methods with deep learning. We summarize the processing pipeline and discuss these methods from four perspectives: deep feature extraction, deep neural network architecture design, personal calibration as well as device and platform. Since the data pre-processing and post-processing methods are crucial for gaze estimation, we also survey face/eye detection method, data rectification method, 2D/3D gaze conversion method, and gaze origin conversion method. To fairly compare the performance of various gaze estimation approaches, we characterize all the publicly available gaze estimation datasets and collect the code of typical gaze estimation algorithms. We implement these codes and set up a benchmark of converting the results of different methods into the same evaluation metrics. This paper not only serves as a reference to develop deep learning-based gaze estimation methods but also a guideline for future gaze estimation research. Implemented methods and data processing codes are available at http://phi-ai.org/GazeHub.

show abstract

Section: A Deep Feature From Appearancementioning

confidence: 99%

Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark

Cheng,

Wang,

Bao

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent work has shown that appearancebased methods can offer good generalization capability even without user-specific data [51]. Thanks to the emerging large-scale data sets and deep learning techniques, the performance of learning-based eye-tracking models has been steadily improving [22,43,47]. Our system unifies eye tracking and video analysis to track human visual attention, allowing video analysis to focus on salient visual content, thereby reducing computation and energy costs.…”

Section: Eye Trackingmentioning

confidence: 99%

MemX

Chang

Zhao

Dong

et al. 2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

This work presents MemX: a biologically-inspired attention-aware eyewear system developed with the goal of pursuing the long-awaited vision of a personalized visual Memex. MemX captures human visual attention on the fly, analyzes the salient visual content, and records moments of personal interest in the form of compact video snippets. Accurate attentive scene detection and analysis on resource-constrained platforms is challenging because these tasks are computation and energy intensive. We propose a new temporal visual attention network that unifies human visual attention tracking and salient visual content analysis. Attention tracking focuses computation-intensive video analysis on salient regions, while video analysis makes human attention detection and tracking more accurate. Using the YouTube-VIS dataset and 30 participants, we experimentally show that MemX significantly improves the attention tracking accuracy over the eye-tracking-alone method, while maintaining high system energy efficiency. We have also conducted 11 in-field pilot studies across a range of daily usage scenarios, which demonstrate the feasibility and potential benefits of MemX.

show abstract

“…Early solutions for gaze-on-screen tracking suffer from low tracking accuracy due to inherent technology limitations and the lack of sufficient training data [5,6]. Recently introduced large-scale data sets, e.g., GazeCapture [7], have greatly enhanced the accuracy of learning-based gaze-onscreen models [8,9,10,11], which capture human face related features, determine the relative mapping between the face and screen coordinate systems, and infer human gaze fixation position on the screen. However, adopting learning-based methods to mobile devices still faces serious performance challenges.…”

Section: Introductionmentioning

confidence: 99%

A High-Frame-Rate Eye-Tracking Framework for Mobile Devices

Chang

Zhao

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Gaze-on-screen tracking, an appearance-based eye-tracking task, has drawn significant interest in recent years. While learning-based high-precision eye-tracking methods have been designed in the past, the complex pre-training and high computation in neural network-based deep models restrict their applicability in mobile devices. Moreover, as the display frame rate of mobile devices has steadily increased to 120 fps, high-frame-rate eye tracking becomes increasingly challenging. In this work, we tackle the tracking efficiency challenge and introduce GazeHFR, a biologic-inspired eyetracking model specialized for mobile devices, offering both high accuracy and efficiency. Specifically, GazeHFR classifies the eye movement into two distinct phases, i.e., saccade and smooth pursuit, and leverages inter-frame motion information combined with lightweight learning models tailored to each movement phase to deliver high-efficient eye tracking without affecting accuracy. Compared to prior art, Gaze-HFR achieves approximately 7x speedup and 15% accuracy improvement on mobile devices. Index Termsgaze estimation, mobile imaging, biomedical video analysis, applications of machine learning * indicates an equal contribution. † indicates the corresponding author

show abstract

Generalizing Eye Tracking With Bayesian Adversarial Learning

Cited by 86 publications

References 36 publications

Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark

Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark

MemX

A High-Frame-Rate Eye-Tracking Framework for Mobile Devices

Contact Info

Product

Resources

About