Mixed Effects Neural Networks (MeNets) With Applications to Gaze Estimation

Xiong, Yan; Kim, Hyunwoo J.; Singh, Vikas

doi:10.1109/cvpr.2019.00793

Cited by 81 publications

(65 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhao et al (Chen and Shi 2018) proposed a CNNs-based method using dilated convolution to estimate gaze from face and eye images. Xiong et al (Xiong and Kim 2019) combines the mixed effects model with CNN and estimates gaze from face images.…”

Section: Appearance-based Methodsmentioning

confidence: 99%

A Coarse-to-Fine Adaptive Network for Appearance-Based Gaze Estimation

Cheng

Huang

Wang

et al. 2020

AAAI

117

View full text Add to dashboard Cite

Human gaze is essential for various appealing applications. Aiming at more accurate gaze estimation, a series of recent works propose to utilize face and eye images simultaneously. Nevertheless, face and eye images only serve as independent or parallel feature sources in those works, the intrinsic correlation between their features is overlooked. In this paper we make the following contributions: 1) We propose a coarse-to-fine strategy which estimates a basic gaze direction from face image and refines it with corresponding residual predicted from eye images. 2) Guided by the proposed strategy, we design a framework which introduces a bi-gram model to bridge gaze residual and basic gaze direction, and an attention component to adaptively acquire suitable fine-grained feature. 3) Integrating the above innovations, we construct a coarse-to-fine adaptive network named CA-Net and achieve state-of-the-art performances on MPIIGaze and EyeDiap.

show abstract

Section: Appearance-based Methodsmentioning

confidence: 99%

A Coarse-to-Fine Adaptive Network for Appearance-Based Gaze Estimation

Cheng

Huang

Wang

et al. 2020

AAAI

117

View full text Add to dashboard Cite

show abstract

“…11). Xiong et al [145] provided mixed effects neural networks (MeNets) which adapts the mixed effects strategy from statistics to a DNN architecture for gaze estimation from eye images. It improves prediction accuracy by 10%-20% on many publicly available benchmarks.…”

Section: Gaze Predictionmentioning

confidence: 99%

VR content creation and exploration with deep learning: A survey

Wang

Lyu

et al. 2020

Comp. Visual Media

View full text Add to dashboard Cite

Virtual reality (VR) offers an artificial, computer generated simulation of a real life environment. It originated in the 1960s and has evolved to provide increasing immersion, interactivity, imagination, and intelligence. Because deep learning systems are able to represent and compose information at various levels in a deep hierarchical fashion, they can build very powerful models which leverage large quantities of visual media data. Intelligence of VR methods and applications has been significantly boosted by the recent developments in deep learning techniques. VR content creation and exploration relates to image and video analysis, synthesis and editing, so deep learning methods such as fully convolutional networks and general adversarial networks are widely employed, designed specifically to handle panoramic images and video and virtual 3D scenes. This article surveys recent research that uses such deep learning methods for VR content creation and exploration. It considers the problems involved, and discusses possible future directions in this active and emerging research area. Keywords virtual reality; deep learning; neural networks; 360 • image and video virtual content

show abstract

“…Krafka et al [24] used the CNN features of a few calibration samples to train a Support Vector Regression model. Several methods [28,7,53] have tried to explicitly incorporate person-dependent parameters in the model. These parameters are estimated with a few samples during testing.…”

Section: Related Workmentioning

confidence: 99%

“…With recent advances in machine learning, the combination of appearance-based gaze estimation and deep learning has become a popular method for remote estimation of gaze [53,7,54,48,13,55]. Appearance-based algorithms have the advantage of only using an image of the person's face or eyes as input, therefore eliminating the need of any specialized hardware other than a regular camera.…”

Section: Introductionmentioning

confidence: 99%

SalGaze: Personalizing Gaze Estimation using Visual Saliency

Chang

Martino

Qiu

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Traditional gaze estimation methods typically require explicit user calibration to achieve high accuracy. This process is cumbersome and recalibration is often required when there are changes in factors such as illumination and pose. To address this challenge, we introduce SalGaze, a framework that utilizes saliency information in the visual content to transparently adapt the gaze estimation algorithm to the user without explicit user calibration. We design an algorithm to transform a saliency map into a differentiable loss map that can be used for the optimization of CNN-based models. SalGaze is also able to greatly augment standard point calibration data with implicit video saliency calibration data using a unified framework. We show accuracy improvements over 24% using our technique on existing methods.

show abstract

Mixed Effects Neural Networks (MeNets) With Applications to Gaze Estimation

Cited by 81 publications

References 38 publications

A Coarse-to-Fine Adaptive Network for Appearance-Based Gaze Estimation

A Coarse-to-Fine Adaptive Network for Appearance-Based Gaze Estimation

VR content creation and exploration with deep learning: A survey

SalGaze: Personalizing Gaze Estimation using Visual Saliency

Contact Info

Product

Resources

About