LatentGaze: Cross-Domain Gaze Estimation Through Gaze-Aware Analytic Latent Code Manipulation

Lee, Isack; Yun, Jun‐Seok; Kim, Hee Hyeon; Na, Youngju; Yoo, Seok Bong

doi:10.1007/978-3-031-26348-4_10

Cited by 9 publications

(9 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gaze SDA (Krafka et al 2016;He et al 2019;Yu, Liu, and Odobez 2019b) and UDA (Kellnhofer et al 2019;Lee et al 2022) methods usually fine-tune the source domain pretrained model on a few labeled or unlabeled target domain samples to improve cross-domain gaze estimation performance, respectively. For example, gaze SDA methods are often based on meta-learning (Park et al 2019), gaze difference (Liu et al 2018(Liu et al , 2019 and gaze decomposition Shi 2020, 2022), while gaze UDA methods are often based on adversarial learning (Wang et al 2019;Lahiri, Agarwalla, and Biswas 2018), teacher-student networks (He et al 2019;Liu et al 2021), representation learning (Guo et al 2021;, rotation consistency (Bao et al 2022) and jitter (Liu et al 2022).…”

Section: Related Workmentioning

confidence: 99%

“…These methods typically use adversarial learning (Cheng, Bao, and Lu 2022) or adversarial attack (Xu, Wang, and Lu 2023) to eliminate or disturb gaze-irrelevant factors or features. Moreover, some gaze SDA, UDA or redirection methods also contain gaze DG modules, such as (Park et al 2019;Bao et al 2022;Lee et al 2022). However, their cross-domain performances still have room for improvement.…”

Section: Related Workmentioning

confidence: 99%

“…For DG, PureGaze (Cheng, Bao, and Lu 2022) and Gaze-Con (Xu, Wang, and Lu 2023) have been proposed. Moreover, some gaze SDA, UDA, redirection, and unconstrained gaze estimation methods contain gaze DG modules, including FAZE (Park et al 2019), RAT (Bao et al 2022), CDG and LatentGaze (Lee et al 2022).…”

Section: Total Loss Functionmentioning

confidence: 99%

“…And we compare our proposed method with Baseline (Zhang et al 2020) (based on ResNet (He et al 2016)). Furthermore, although our proposed gaze DG method is not designed for gaze UDA, we also compare our proposed method with other gaze UDA methods for reference, including ADDA (Tzeng et al 2017), GazeAdv (Wang et al 2019), Gaze360 (Kellnhofer et al 2019), DAGEN (Guo et al 2021), PnP-GA (Liu et al 2021), RUDA (Bao et al 2022), CRGA and LatentGaze (Lee et al 2022).…”

Section: Total Loss Functionmentioning

confidence: 99%

See 3 more Smart Citations

Gaze from Origin: Learning for Generalized Gaze Estimation by Embedding the Gaze Frontalization Process

Xu,

2024

AAAI

View full text Add to dashboard Cite

Gaze estimation aims to accurately estimate the direction or position at which a person is looking. With the development of deep learning techniques, a number of gaze estimation methods have been proposed and achieved state-of-the-art performance. However, these methods are limited to within-dataset settings, whose performance drops when tested on unseen datasets. We argue that this is caused by infinite and continuous gaze labels. To alleviate this problem, we propose using gaze frontalization as an auxiliary task to constrain gaze estimation. Based on this, we propose a novel gaze domain generalization framework named Gaze Frontalization-based Auxiliary Learning (GFAL) Framework which embeds the gaze frontalization process, i.e., guiding the feature so that the eyeball can rotate and look at the front (camera), without any target domain information during training. Experimental results show that our proposed framework is able to achieve state-of-the-art performance on gaze domain generalization task, which is competitive with or even superior to the SOTA gaze unsupervised domain adaptation methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Total Loss Functionmentioning

confidence: 99%

Section: Total Loss Functionmentioning

confidence: 99%

See 2 more Smart Citations

Gaze from Origin: Learning for Generalized Gaze Estimation by Embedding the Gaze Frontalization Process

Xu,

2024

AAAI

View full text Add to dashboard Cite

show abstract

“…For DG methods, we choose CDG (Wang et al 2022), PureGaze and Xu et al's method (Xu, Wang, and Lu 2023) for comparison and use the results report by the author. Additionally, the results of SOTA UDA methods, including PnP-GA (Liu et al 2021), RUDA (Bao et al 2022), CRGA (Wang et al 2022), LatentGaze (Lee et al 2022), Liu et al's work (Liu et al 2022) and UnReGA (Cai et al 2023) as a reference.…”

Section: Experiments Experiments Detailsmentioning

confidence: 99%

CLIP-Gaze: Towards General Gaze Estimation via Visual-Linguistic Model

Yin,

Zeng,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

Gaze estimation methods often experience significant performance degradation when evaluated across different domains, due to the domain gap between the testing and training data. Existing methods try to address this issue using various domain generalization approaches, but with little success because of the limited diversity of gaze datasets, such as appearance, wearable, and image quality. To overcome these limitations, we propose a novel framework called CLIP-Gaze that utilizes a pre-trained vision-language model to leverage its transferable knowledge. Our framework is the first to leverage the vision-and-language cross-modality approach for gaze estimation task. Specifically, we extract gaze-relevant feature by pushing it away from gaze-irrelevant features which can be flexibly constructed via language descriptions. To learn more suitable prompts, we propose a personalized context optimization method for text prompt tuning. Furthermore, we utilize the relationship among gaze samples to refine the distribution of gaze-relevant features, thereby improving the generalization capability of the gaze estimation model. Extensive experiments demonstrate the excellent performance of CLIP-Gaze over existing methods on four cross-domain evaluations.

show abstract

Cascaded learning with transformer for simultaneous eye landmark, eye state and gaze estimation

Gou,

Yu,

Guo

et al. 2024

Pattern Recognition

View full text Add to dashboard Cite

LatentGaze: Cross-Domain Gaze Estimation Through Gaze-Aware Analytic Latent Code Manipulation

Cited by 9 publications

References 27 publications

Gaze from Origin: Learning for Generalized Gaze Estimation by Embedding the Gaze Frontalization Process

Gaze from Origin: Learning for Generalized Gaze Estimation by Embedding the Gaze Frontalization Process

CLIP-Gaze: Towards General Gaze Estimation via Visual-Linguistic Model

Cascaded learning with transformer for simultaneous eye landmark, eye state and gaze estimation

Contact Info

Product

Resources

About