A Deep Learning Approach to Appearance-Based Gaze Estimation under Head Pose Variations

Sun, Hsin-Pei; Yang, Cheng-Hsun; Lai, Shang-Hong

doi:10.1109/acpr.2017.155

“…1.7 presenting a system that locates the face and eye positions, then applying a modified appearance-based technique for face and eye detection [31]. Another example of a head-pose and gaze estimation is presented in [32], which shows the whole training and estimation pipeline of their system using multimodal convolutional neural networks for appearance-based gaze estimation. This thesis is concerned with exploring and finding new features that are more robust and accurate than the existing methods in literature.…”

Section: Objectivementioning

confidence: 99%

“…For instance, the case Skx and Sky are zero indicates symmetry. If both are positive, the distribution is skewed left of the y axis and below the x axis [32].…”

Section: Geometrical Momentsmentioning

confidence: 99%

Robust Discriminative Analysis Framework for Gaze and Headpose Estimation

Rabba¹

2021

Preprint

0

View full text Add to dashboard Cite

Head movements, combined with gaze, play a fundamental role in predicting a person’s action and intention. In non-constrained head movement settings, the process is complex, and performance can degrade significantly in the presence of variation in head-pose, gaze position, occlusion and ambient illumination. In this thesis, a framework is therefore proposed to fuse and combine head-pose and gaze information to obtain more robust and accurate gaze estimation. Specific contributions include: the development of a newly developed graph-based model for pupil localization and accurate estimation of the pupil center; the proposal of a novel iris region descriptor feature using quadtree decomposition, that works together with pupil localization for gaze estimation; the proposal of kernel-based extensions and enhancements to a fusion mechanism known as Discriminative Multiple Canonical Correlation Analysis (DMCCA) for fusing features (proposed and traditional) together, to generate a refined, high quality feature set for classification; and the newly developed methodology of head-pose features based on quadtree decompositions and geometrical moments, to better integrate roll, yaw, pitch and jawline into the overall estimation framework. The experimental results of the proposed framework demonstrate robustness against variations in illumination, occlusion, head-pose and is calibration free. The proposed framework was validated on several datasets and scored: 4.5° using MPII, 4.4° using Cave, 4.8° using EYEDIAP, 5.0° using ACS, 4.1° using OSLO and 4.5° using UULM datasets respectively.

show abstract

“…1.7 presenting a system that locates the face and eye positions, then applying a modified appearance-based technique for face and eye detection [31]. Another example of a head-pose and gaze estimation is presented in [32], which shows the whole training and estimation pipeline of their system using multimodal convolutional neural networks for appearance-based gaze estimation. This thesis is concerned with exploring and finding new features that are more robust and accurate than the existing methods in literature.…”

Section: Objectivementioning

confidence: 99%

“…For instance, the case Skx and Sky are zero indicates symmetry. If both are positive, the distribution is skewed left of the y axis and below the x axis [32].…”

Section: Geometrical Momentsmentioning

confidence: 99%

Robust Discriminative Analysis Framework for Gaze and Headpose Estimation

Rabba¹

2021

Preprint

0

View full text Add to dashboard Cite

Head movements, combined with gaze, play a fundamental role in predicting a person’s action and intention. In non-constrained head movement settings, the process is complex, and performance can degrade significantly in the presence of variation in head-pose, gaze position, occlusion and ambient illumination. In this thesis, a framework is therefore proposed to fuse and combine head-pose and gaze information to obtain more robust and accurate gaze estimation. Specific contributions include: the development of a newly developed graph-based model for pupil localization and accurate estimation of the pupil center; the proposal of a novel iris region descriptor feature using quadtree decomposition, that works together with pupil localization for gaze estimation; the proposal of kernel-based extensions and enhancements to a fusion mechanism known as Discriminative Multiple Canonical Correlation Analysis (DMCCA) for fusing features (proposed and traditional) together, to generate a refined, high quality feature set for classification; and the newly developed methodology of head-pose features based on quadtree decompositions and geometrical moments, to better integrate roll, yaw, pitch and jawline into the overall estimation framework. The experimental results of the proposed framework demonstrate robustness against variations in illumination, occlusion, head-pose and is calibration free. The proposed framework was validated on several datasets and scored: 4.5° using MPII, 4.4° using Cave, 4.8° using EYEDIAP, 5.0° using ACS, 4.1° using OSLO and 4.5° using UULM datasets respectively.

show abstract