Controlling gaze with an embodied interactive control architecture

Mohammad, Yasser; Nishida, Toyoaki

doi:10.1007/s10489-009-0180-0

Cited by 17 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gazing is a crucial form of human behavioral information that contains a wealth of psychological insights and is a vital clue for comprehending human intentions and emotions [1]. Gaze estimation by the human eye has a wide range of applications, including medical treatment [2,3], virtual reality [4], human-computer interaction [5][6][7][8], market research [9], and other fields. Gaze estimation research can be broadly categorized into three areas based on different scenarios and applications: gaze point prediction [10], gaze target prediction [11], and three-dimensional gaze estimation [12,13].…”

Section: Introductionmentioning

confidence: 99%

Gaze Estimation Based on Convolutional Structure and Sliding Window-Based Attention Mechanism

Chen

et al. 2023

Sensors

View full text Add to dashboard Cite

The direction of human gaze is an important indicator of human behavior, reflecting the level of attention and cognitive state towards various visual stimuli in the environment. Convolutional neural networks have achieved good performance in gaze estimation tasks, but their global modeling capability is limited, making it difficult to further improve prediction performance. In recent years, transformer models have been introduced for gaze estimation and have achieved state-of-the-art performance. However, their slicing-and-mapping mechanism for processing local image patches can compromise local spatial information. Moreover, the single down-sampling rate and fixed-size tokens are not suitable for multiscale feature learning in gaze estimation tasks. To overcome these limitations, this study introduces a Swin Transformer for gaze estimation and designs two network architectures: a pure Swin Transformer gaze estimation model (SwinT-GE) and a hybrid gaze estimation model that combines convolutional structures with SwinT-GE (Res-Swin-GE). SwinT-GE uses the tiny version of the Swin Transformer for gaze estimation. Res-Swin-GE replaces the slicing-and-mapping mechanism of SwinT-GE with convolutional structures. Experimental results demonstrate that Res-Swin-GE significantly outperforms SwinT-GE, exhibiting strong competitiveness on the MpiiFaceGaze dataset and achieving a 7.5% performance improvement over existing state-of-the-art methods on the Eyediap dataset.

show abstract

Section: Introductionmentioning

confidence: 99%

Gaze Estimation Based on Convolutional Structure and Sliding Window-Based Attention Mechanism

Chen

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…It contains rich information about human features, allowing researchers and users to tap more about human patterns [ 1 , 2 ] and action [ 3 , 4 ]. It is widely recommended in many topics, e.g., human–robot interaction (HRI) [ 5 , 6 , 7 , 8 ]. Most common gaze estimation tasks are categorized into three types: (1) three-dimensional (3D)-based gaze estimation [ 9 ], (2) target estimation [ 10 , 11 ] and (3) tracking estimation [ 12 ].…”

Section: Introductionmentioning

confidence: 99%

Gaze Estimation Approach Using Deep Differential Residual Network

Huang

Wang

et al. 2022

Sensors

View full text Add to dashboard Cite

Gaze estimation, which is a method to determine where a person is looking at given the person’s full face, is a valuable clue for understanding human intention. Similarly to other domains of computer vision, deep learning (DL) methods have gained recognition in the gaze estimation domain. However, there are still gaze calibration problems in the gaze estimation domain, thus preventing existing methods from further improving the performances. An effective solution is to directly predict the difference information of two human eyes, such as the differential network (Diff-Nn). However, this solution results in a loss of accuracy when using only one inference image. We propose a differential residual model (DRNet) combined with a new loss function to make use of the difference information of two eye images. We treat the difference information as auxiliary information. We assess the proposed model (DRNet) mainly using two public datasets (1) MpiiGaze and (2) Eyediap. Considering only the eye features, DRNet outperforms the state-of-the-art gaze estimation methods with angular-error of 4.57 and 6.14 using MpiiGaze and Eyediap datasets, respectively. Furthermore, the experimental results also demonstrate that DRNet is extremely robust to noise images.

show abstract

“…It contains rich information about human features, allowing researchers and users to tap more about human patterns [1,2] and action [3,4]. It is widely recommended in many topics, e.g., human-robot interaction (HRI) [5,6,7,8]. Most common gaze estimation tasks are categorized into three types: (1) three-dimensional (3D)-based gaze estimation [9], (2) target estimation [10,11] and (3) tracking estimation [12].…”

Section: Introductionmentioning

confidence: 99%

Gaze Estimation Approach Using Deep Differential Residual Network

Huang,

Li,

Wang

et al. 2022

Preprint

View full text Add to dashboard Cite

Gaze estimation, which is a method to determine where a person is looking at given the person's full face, is a valuable clue for understanding human intention. Similarly to other domains of computer vision, deep learning (DL) methods have gained recognition in the gaze estimation domain. However, there are still gaze calibration problems in the gaze estimation domain, thus preventing existing methods from further improving the performances. An effective solution is to directly predict the difference information of two human eyes, such as the differential network (Diff-Nn). However, this solution results in a loss of accuracy when using only one inference image. We propose a differential residual model (DRNet) combined with a new loss function to make use of the difference information of two eye images. We treat the difference information as auxiliary information. We assess the proposed model (DRNet) mainly using two public datasets (1) MpiiGaze and (2) Eyediap. Considering only the eye features, DRNet outperforms the state-of-the-art gaze estimation methods with angular − error of 4.57 and 6.14 using MpiiGaze and Eyediap datasets, respectively. Furthermore, the experimental results also demonstrate that DRNet is extremely robust to noise images.

show abstract

Controlling gaze with an embodied interactive control architecture

Cited by 17 publications

References 15 publications

Gaze Estimation Based on Convolutional Structure and Sliding Window-Based Attention Mechanism

Gaze Estimation Based on Convolutional Structure and Sliding Window-Based Attention Mechanism

Gaze Estimation Approach Using Deep Differential Residual Network

Gaze Estimation Approach Using Deep Differential Residual Network

Contact Info

Product

Resources

About