Impact of mask use on face recognition: an eye-tracking study

Abstract

We examined how mask use affects performance and eye movements in face recognition and whether strategy change reflected in eye movements is associated with performance change. Eighty-eight participants performed face recognition with masked faces either during learning only, during recognition only, or during both learning and recognition. As compared with the baseline condition where faces were unmasked during both learning and recognition, participants had impaired performance in all three scenarios, with larger impairment when mask conditions during learning and recognition did not match. When recognizing unmasked faces, whether the faces were learned with or without a mask on did not change eye movement behavior. Nevertheless, when recognizing unmasked faces that were learned with a mask on, participants who adopted more eyes-focused patterns had less performance impairment as compared with the baseline condition. When recognizing masked faces, participants had more eyes-focused patterns and more consistent gaze transition behavior than recognizing unmasked faces regardless of whether the faces were learned with or without a mask on. Nevertheless, when recognizing masked faces that were learned without a mask, participants whose gaze transition behavior was more consistent had less performance impairment as compared with the baseline condition. Thus, although eye movements during recognition were mainly driven by the mask condition during recognition but not that during learning, those who adjusted their strategy according to the mask condition difference between learning and recognition had better performance. This finding has important implications for identifying populations vulnerable to the impact of mask use and potential remedial strategies.

How does face mask in COVID-19 pandemic disrupt face learning and recognition in adults with autism spectrum disorder?

“…In Study 2, we found that individuals with ASD had poorer performance than typical participants in the recognition of unmasked faces that were learned with a masked on. Hsiao et al ( 2022 ) showed that in this scenario, typical adults who adjusted their face-recognition strategies by shifting their gazes toward the eyes of the unmasked faces had better recognition performance. This shift in eye gaze strategy toward the eyes is beneficial in this scenario since these unmasked faces were learned with a mask on, revealing only the information around the eye region for memory encoding.…”

“…In other words, their strategy change during recognition would be mainly guided by the mask and rely less on cognitive flexibility or interference control, and thus no significant performance difference was observed between the two participant groups. Hsiao et al ( 2022 ) showed that in this scenario, typical adults whose eye fixation behavior was more consistent across trials had better performance, and this eye fixation behavior consistency was particularly correlated with non-verbal IQ as measured in the nine-item Raven’s test. Since in the current study the ASD and typical groups were matched in non-verbal IQ, the result that they did not differ in performance in this scenario was consistent with Hsiao et al’s finding ( 2022 ).…”

“…Hsiao et al ( 2022 ) showed that in this scenario, typical adults whose eye fixation behavior was more consistent across trials had better performance, and this eye fixation behavior consistency was particularly correlated with non-verbal IQ as measured in the nine-item Raven’s test. Since in the current study the ASD and typical groups were matched in non-verbal IQ, the result that they did not differ in performance in this scenario was consistent with Hsiao et al’s finding ( 2022 ). Note however that although IQ is related to executive functioning, ASD individuals may still differ in their executive functioning skills from the typical adults when IQ was matched (Benallie et al, 2021 ), and this potential executive functioning difference might be related to the performance difference observed in Study 2 when recognizing unmasked faces that were learned with a mask on.…”

HSI: Human Saliency Imitator for Benchmarking Saliency-Based Model Explanations

“…For example, Hwu et al (2021) compared heatmap-based XAI methods, LRP, with human attention data and reported a higher similarity to task-driven attentive human attention than inattentive attention. Note however that cogntive science research has consistently shown that human attention during image viewing are both task-specific (e.g., Borji and Itti 2015;Kanan et al 2015) and person-specific (e.g., Hsiao et al 2021a;An and Hsiao 2021;Hsiao et al 2021b;Hsiao, Liao, and Tso 2022). As the purpose of XAI is to provide explanations, comparing saliency-based XAI with human attention that are associated with better performance during an explanation task will provide more insights into the quality of XAI.…”

Differential audiovisual information processing in emotion recognition: An eye-tracking study.

“…To measure participants' eyemovement pattern in each condition along the dimension of the two representative group patterns, we defined A-B scale as A-B Scale = (A -B)/(jAj þ jBj), where A stands for the log-likelihood of the participant's eye-movement data being classified as the first pattern and B stands for the log-likelihood of the data being classified as the second pattern. A more positive A-B scale value indicates higher similarity to the first pattern as opposed to the second pattern (e.g.,Chan et al, 2018;Hsiao et al, 2022).Transparency and OpennessWe report how we determined our sample size, all manipulations, and all measures in the study. Data were analyzed using SPSS(Nie et al, 1975) and Jamovi(S ahin & Aybek, 2019).…”