Extracting discriminative features using task-oriented gaze maps measured from observers for personal attribute classification

Nishiyama, Masashi; Matsumoto, Riku; Yoshimura, Hideyuki; Iwai, Yoshio

doi:10.1016/j.patrec.2018.08.001

Cited by 8 publications

(5 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We see that the participants mainly viewed the head regions of the subjects in the stimulus images. This tendency was identical to that reported in a past study [12]. We think that regions with large values in gaze distribution g contained informative features because these regions were attended to by the participants when judging gender.…”

Section: Measurement Of the Gaze Distributionsupporting

confidence: 89%

“…To avoid this expense, we measure a representative gaze distribution from a few stimulus images. In our experiments, the gaze measurement was performed according to the procedure described in [12]. The details of the gaze measurement are described below.…”

Section: Measurement Of the Gaze Distributionmentioning

confidence: 99%

“…Researchers [7]- [12] have recently attempted to extract informative features for classification tasks by incorporating the gaze distribution (such as fixation and saccade), where this distribution is measured when observers view images. However, they have not considered the background bias of the training samples.…”

Section: Introductionmentioning

confidence: 99%

“…We think, nonetheless, that the use of the gaze distribution can help solve the problem of background bias. In particular, Nishiyama et al [12] proposed a gender recognition method using the gaze distribution for pedestrian images. Their method involves measuring the distribution of gaze locations when observers judge gender in pedestrian images and then using the distribution in prepossessing to extract discriminative features.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Gender Recognition Using a Gaze-Guided Self-Attention Mechanism Robust Against Background Bias in Training Samples

Nishiyama

Inoue

Iwai

2022

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

We propose an attention mechanism in deep learning networks for gender recognition using the gaze distribution of human observers when they judge the gender of people in pedestrian images. Prevalent attention mechanisms spatially compute the correlation among values of all cells in an input feature map to calculate attention weights. If a large bias in the background of pedestrian images (e.g., test samples and training samples containing different backgrounds) is present, the attention weights learned using the prevalent attention mechanisms are affected by the bias, which in turn reduces the accuracy of gender recognition. To avoid this problem, we incorporate an attention mechanism called gaze-guided selfattention (GSA) that is inspired by human visual attention. Our method assigns spatially suitable attention weights to each input feature map using the gaze distribution of human observers. In particular, GSA yields promising results even when using training samples with the background bias. The results of experiments on publicly available datasets confirm that our GSA, using the gaze distribution, is more accurate in gender recognition than currently available attention-based methods in the case of background bias between training and test samples.

show abstract

Section: Measurement Of the Gaze Distributionsupporting

confidence: 89%

Section: Measurement Of the Gaze Distributionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Gender Recognition Using a Gaze-Guided Self-Attention Mechanism Robust Against Background Bias in Training Samples

Nishiyama

Inoue

Iwai

2022

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Huang [18] fuses the visual saliency and task-dependent in eye-tracking gaze to learn eye attention transfer and predict gaze information. Related studies [19][20][21][22][23] on gaze point prediction take into account the integration of visual saliency and human gaze attention transfer, that is, the underlying visual characteristics of eye gaze visual characteristics and human brain consciousness attention. The similar research on Named entity recognition studies [24][25][26][27] need to encode input features and output recognition sequences, these are applicated in weapon, product quality and language areas.…”

Section: Introductionmentioning

confidence: 99%

A Study of Eye-Tracking Gaze Point Classification and Application Based on Conditional Random Field

Bai

Wang

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

The head-mounted eye-tracking technology is often used to manipulate the motion of servo platform in remote tasks, so as to achieve visual aiming of servo platform, which is a highly integrated human-computer interaction effect. However, it is difficult to achieve accurate manipulation for the uncertain meanings of gaze points in eye-tracking. To solve this problem, a method of classifying gaze points based on a conditional random field is proposed. It first describes the features of gaze points and gaze images, according to the eye visual characteristic. An LSTM model is then introduced to merge these two features. Afterwards, the merge features are learned by CRF model to obtain the classified gaze points. Finally, the meaning of gaze point is classified for target, in order to accurately manipulate the servo platform. The experimental results show that the proposed method can classify more accurate target gaze points for 100 images, the average evaluation values Precision = 86.81%, Recall = 86.79%, We = 86.79%, these are better than relevant methods. In addition, the isolated gaze points can be eliminated, and the meanings of gaze points can be classified to achieve the accuracy of servo platform visual aiming.

show abstract

Body-Part Attention Probability for Measuring Gaze During Impression Word Evaluation

Kinoshita

Inoue

Nishiyama

et al. 2021

HCI International 2021 - Posters

View full text Add to dashboard Cite

Extracting discriminative features using task-oriented gaze maps measured from observers for personal attribute classification

Cited by 8 publications

References 41 publications

Gender Recognition Using a Gaze-Guided Self-Attention Mechanism Robust Against Background Bias in Training Samples

Gender Recognition Using a Gaze-Guided Self-Attention Mechanism Robust Against Background Bias in Training Samples

A Study of Eye-Tracking Gaze Point Classification and Application Based on Conditional Random Field

Body-Part Attention Probability for Measuring Gaze During Impression Word Evaluation

Contact Info

Product

Resources

About