Guiding Intelligent Surveillance System by learning-by-synthesis gaze estimation

Zhao, Tongtong; Yan, Yuxiao; Peng, Jinjia; Mi, Zetian; Fu, Xianping

doi:10.48550/arxiv.1810.03286

Cited by 2 publications

(2 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First method is a simple cascaded method [8][32] [30] which uses multiple k-NN(k-Nearest Neighbor) classifier to select neighbors in feature space joint head pose,pupil center and eye appearance. The other method is to train a simple convolutional neural network (CNN) [11][29] [31] to predict the eye gaze direction with l 2 loss. We train on UnityEyes ,UTView, SynthesEyes and test on MPIIGaze, purified MPIIGaze which is purified by proposed method.…”

Section: Appearance-based Gaze Estimationmentioning

confidence: 99%

Purifying Real Images with an Attention-guided Style Transfer Network for Gaze Estimation

Yan¹,

Yang²,

Peng³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Recently, the progress of learning-by-synthesis has proposed a training model for synthetic images, which can effectively reduce the cost of human and material resources. However, due to the different distribution of synthetic images compared to real images, the desired performance cannot be achieved. Real images consist of multiple forms of light orientation, while synthetic images consist of a uniform light orientation. These features are considered to be characteristic of outdoor and indoor scenes, respectively. To solve this problem, the previous method learned a model to improve the realism of the synthetic image. Different from the previous methods, this paper try to purify real image by extracting discriminative and robust features to convert outdoor real images to indoor synthetic images. In this paper, we first introduce the segmentation masks to construct RGB-mask pairs as inputs, then we design a attention-guided style transfer network to learn style features separately from the attention and bkgd(background) region , learn content features from full and attention region. Moreover, we propose a novel region-level task-guided loss to restrain the features learnt from style and content. Experiments were performed using mixed studies (qualitative and quantitative) methods to demonstrate the possibility of purifying real images in complex directions. We evaluate the proposed method on three public datasets, including LPW, COCO and MPIIGaze. Extensive experimental results show that the proposed method is effective and achieves the state-of- * Xianping Fu is corresponding author.

show abstract

Section: Appearance-based Gaze Estimationmentioning

confidence: 99%

Purifying Real Images with an Attention-guided Style Transfer Network for Gaze Estimation

Yan¹,

Yang²,

Peng³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…First method is a simple cascaded method [9][37] [35] which uses multiple k-NN(k-Nearest Neighbor) classifier to select neighbors in feature space joint head pose,pupil center and eye appearance. The other method is to train a simple convolutional neural network (CNN) [12][34] [36] to predict the eye gaze direction with l 2 loss. We train on UnityEyes ,UTView, SynthesEyes and test on MPIIGaze, purified MPIIGaze which is purified by proposed method.…”

Section: Appearance-based Gaze Estimationmentioning

confidence: 99%

Purifying naturalistic images through a real-time style transfer semantics network

Zhao

Yan

Shehu

et al. 2019

Engineering Applications of Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Recently, the progress of learning-by-synthesis has proposed a training model for synthetic images, which can effectively reduce the cost of human and material resources. However, due to the different distribution of synthetic images compared to real images, the desired performance cannot still be achieved. Real images consist of multiple forms of light orientation, while synthetic images consist of a uniform light orientation. These features are considered to be characteristic of outdoor and indoor scenes, respectively. To solve this problem, the previous method learned a model to improve the realism of the synthetic image. Different from the previous methods, this paper takes the first step to purify real images. Through the style transfer task, the distribution of outdoor real images is converted into indoor synthetic images, thereby reducing the influence of light. Therefore, this paper proposes a real-time style transfer network that preserves image content information (eg, gaze direction, pupil center position) of an input image (real image) while inferring style information (eg, image color structure, semantic features) of style image (synthetic image). In addition, the network accelerates the convergence speed of the model and adapts to multi-scale images. Experiments were performed using mixed studies (qualitative and quantitative) methods to demonstrate the possibility of purifying real images in complex directions. Qualitatively, it compares the proposed method with the available methods * Both Xianping Fu and Huibing Wang are corresponding authors.in a series of indoor and outdoor scenarios of the LPW dataset. In quantitative terms, it evaluates the purified image by training a gaze estimation model on the cross data set. The results show a significant improvement over the baseline method compared to the raw real image.

show abstract

Guiding Intelligent Surveillance System by learning-by-synthesis gaze estimation

Cited by 2 publications

References 25 publications

Purifying Real Images with an Attention-guided Style Transfer Network for Gaze Estimation

Purifying Real Images with an Attention-guided Style Transfer Network for Gaze Estimation

Purifying naturalistic images through a real-time style transfer semantics network

Contact Info

Product

Resources

About