Learning a gaze estimator with neighbor selection from large-scale synthetic eye images

Zhao, Tongtong; Ding, Xueyan; Peng, Jinjia; Bian, Jiming; Fu, Xianping

doi:10.1016/j.knosys.2017.10.010

Cited by 26 publications

(18 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Implementation Details: In order to verify the effectiveness of the proposed method for gaze estimation, 3 public datasets (UTView [4], SynthesEyes [5], UnityEyes [9]) are used to train the estimator with k-NN [8], MPIIGaze dataset [10] and purified MPIIGaze dataset (purified by proposed method) are used for test the accuracy. The eye gaze estimation network is similar to [29][31], the input is a 35 × 55 gray scale image that is passed through 5 convolutional layers followed by 3 fully connected layers, the last one encoding the 3-dimensional gaze vector: (1)Conv 32@3×3 (2)Conv 32@3×3 (3)Conv 64@3×3 (4)Max-Pooling 3×3 (5)Conv 80@3×3 (6)Conv 192@3×3 (7)Max-Pooling 2×2 (8)FC9600 ( 9)FC1000 ( 10)FC3 (11)Euclidean loss.…”

Section: Appearance-based Gaze Estimationmentioning

confidence: 99%

Purifying Real Images with an Attention-guided Style Transfer Network for Gaze Estimation

Yan¹,

Yang²,

Peng³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Recently, the progress of learning-by-synthesis has proposed a training model for synthetic images, which can effectively reduce the cost of human and material resources. However, due to the different distribution of synthetic images compared to real images, the desired performance cannot be achieved. Real images consist of multiple forms of light orientation, while synthetic images consist of a uniform light orientation. These features are considered to be characteristic of outdoor and indoor scenes, respectively. To solve this problem, the previous method learned a model to improve the realism of the synthetic image. Different from the previous methods, this paper try to purify real image by extracting discriminative and robust features to convert outdoor real images to indoor synthetic images. In this paper, we first introduce the segmentation masks to construct RGB-mask pairs as inputs, then we design a attention-guided style transfer network to learn style features separately from the attention and bkgd(background) region , learn content features from full and attention region. Moreover, we propose a novel region-level task-guided loss to restrain the features learnt from style and content. Experiments were performed using mixed studies (qualitative and quantitative) methods to demonstrate the possibility of purifying real images in complex directions. We evaluate the proposed method on three public datasets, including LPW, COCO and MPIIGaze. Extensive experimental results show that the proposed method is effective and achieves the state-of- * Xianping Fu is corresponding author.

show abstract

Section: Appearance-based Gaze Estimationmentioning

confidence: 99%

Purifying Real Images with an Attention-guided Style Transfer Network for Gaze Estimation

Yan¹,

Yang²,

Peng³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…As shown in Figure 7, the proposed neighbor selection method consists of a double k -NN query in different feature spaces. This work provides a simple version of our previous work [37]. Here, Raw features are used as the appearance descriptor.…”

Section: Proposed Systemmentioning

confidence: 99%

Continuous Driver’s Gaze Zone Estimation Using RGB-D Camera

Yuan

Peng

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

The driver gaze zone is an indicator of a driver’s attention and plays an important role in the driver’s activity monitoring. Due to the bad initialization of point-cloud transformation, gaze zone systems using RGB-D cameras and ICP (Iterative Closet Points) algorithm do not work well under long-time head motion. In this work, a solution for a continuous driver gaze zone estimation system in real-world driving situations is proposed, combining multi-zone ICP-based head pose tracking and appearance-based gaze estimation. To initiate and update the coarse transformation of ICP, a particle filter with auxiliary sampling is employed for head state tracking, which accelerates the iterative convergence of ICP. Multiple templates for different gaze zone are applied to balance the templates revision of ICP under large head movement. For the RGB information, an appearance-based gaze estimation method with two-stage neighbor selection is utilized, which treats the gaze prediction as the combination of neighbor query (in head pose and eye image feature space) and linear regression (between eye image feature space and gaze angle space). The experimental results show that the proposed method outperforms the baseline methods on gaze estimation, and can provide a stable head pose tracking for driver behavior analysis in real-world driving scenarios.

show abstract

“…In addition to common methods such as Support Vector Regression(SVR), Adaptive Linear Regression(ALR) and Random Forest(RF), two methods are reproduced for fairly comparison with stateof-the-art. First method is a simple cascaded method [9][37] [35] which uses multiple k-NN(k-Nearest Neighbor) classifier to select neighbors in feature space joint head pose,pupil center and eye appearance. The other method is to train a simple convolutional neural network (CNN) [12][34] [36] to predict the eye gaze direction with l 2 loss.…”

Section: Appearance-based Gaze Estimationmentioning

confidence: 99%

Purifying naturalistic images through a real-time style transfer semantics network

Zhao

Yan

Shehu

et al. 2019

Engineering Applications of Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Recently, the progress of learning-by-synthesis has proposed a training model for synthetic images, which can effectively reduce the cost of human and material resources. However, due to the different distribution of synthetic images compared to real images, the desired performance cannot still be achieved. Real images consist of multiple forms of light orientation, while synthetic images consist of a uniform light orientation. These features are considered to be characteristic of outdoor and indoor scenes, respectively. To solve this problem, the previous method learned a model to improve the realism of the synthetic image. Different from the previous methods, this paper takes the first step to purify real images. Through the style transfer task, the distribution of outdoor real images is converted into indoor synthetic images, thereby reducing the influence of light. Therefore, this paper proposes a real-time style transfer network that preserves image content information (eg, gaze direction, pupil center position) of an input image (real image) while inferring style information (eg, image color structure, semantic features) of style image (synthetic image). In addition, the network accelerates the convergence speed of the model and adapts to multi-scale images. Experiments were performed using mixed studies (qualitative and quantitative) methods to demonstrate the possibility of purifying real images in complex directions. Qualitatively, it compares the proposed method with the available methods * Both Xianping Fu and Huibing Wang are corresponding authors.in a series of indoor and outdoor scenarios of the LPW dataset. In quantitative terms, it evaluates the purified image by training a gaze estimation model on the cross data set. The results show a significant improvement over the baseline method compared to the raw real image.

show abstract

Learning a gaze estimator with neighbor selection from large-scale synthetic eye images

Cited by 26 publications

References 11 publications

Purifying Real Images with an Attention-guided Style Transfer Network for Gaze Estimation

Purifying Real Images with an Attention-guided Style Transfer Network for Gaze Estimation

Continuous Driver’s Gaze Zone Estimation Using RGB-D Camera

Purifying naturalistic images through a real-time style transfer semantics network

Contact Info

Product

Resources

About