Dynamic human and avatar facial expressions elicit differential brain responses

Abstract

Computer-generated characters, so-called avatars, are widely used in advertising, entertainment, human–computer interaction or as research tools to investigate human emotion perception. However, brain responses to avatar and human faces have scarcely been studied to date. As such, it remains unclear whether dynamic facial expressions of avatars evoke different brain responses than dynamic facial expressions of humans. In this study, we designed anthropomorphic avatars animated with motion tracking and tested whether the human brain processes fearful and neutral expressions in human and avatar faces differently. Our fMRI results showed that fearful human expressions evoked stronger responses than fearful avatar expressions in the ventral anterior and posterior cingulate gyrus, the anterior insula, the anterior and posterior superior temporal sulcus, and the inferior frontal gyrus. Fearful expressions in human and avatar faces evoked similar responses in the amygdala. We did not find different responses to neutral human and avatar expressions. Our results highlight differences, but also similarities in the processing of fearful human expressions and fearful avatar expressions even if they are designed to be highly anthropomorphic and animated with motion tracking. This has important consequences for research using dynamic avatars, especially when processes are investigated that involve cortical and subcortical regions.

The Effects of Dynamic and Static Emotional Facial Expressions of Humans and Their Avatars on the EEG: An ERP and ERD/ERS Study

“…On the other hand, Sarkheil et al (2013) concluded that even when using advanced motion tracking and visualization techniques, the artificial facial motion of avatars leads to differences in the activation of these regions when compared to human facial motion. Taken together Kegel et al (2020) concluded, that dynamic avatar expressions evoke fast amygdala responses comparable to those of humans, but the artificial facial motion of those avatars lead to differences in slower cortical process mechanisms.…”

“…The same stimuli material was used in a successfully published event-related fMRI work by Kegel et al (2020). This fMRI study aimed to investigate whether there are differences in cortical and subcortical processing mechanisms when comparing human and avatar dynamic facial expressions (Kegel et al, 2020).…”

“…The same stimuli material was used in a successfully published event-related fMRI work by Kegel et al (2020). This fMRI study aimed to investigate whether there are differences in cortical and subcortical processing mechanisms when comparing human and avatar dynamic facial expressions (Kegel et al, 2020). Interestingly, their results showed no significant response difference when comparing dynamic fearful expressions between human and avatar faces for subcortical core regions involved in face and emotion perception, like the amygdala.…”

Learning whom not to trust across early adolescence: a longitudinal neuroimaging study to trusting behavior involving an uncooperative other

“…We chose this approach to avoid the need to deceive participants into thinking they were playing the game with a real human partner. In prior studies, participants have been found to respond similarly to human and anthropomorphic characters on a behavioral level, but it should be noted that there are indications that responses to human interaction partners on a neural level are stronger than those to andromorphic characters (e.g., Kegel et al, 2020;Riedl et al, 2011). Nevertheless, findings of the current studies are in line with prior Trust Game and social decision-making studies with human partners (e.g., Fett et al, 2014a;Lemmers-Jansen et al, 2017;Lemmers-Jansen et al, 2019;van den Bos et al, 2011;van Hoorn et al, 2019), suggesting that participants perceived the trustee as a human(-like) being.…”

Preliminary Study of the Perception of Emotions Expressed by Virtual Agents in the Context of Parkinson's Disease

“…They reported that perception of emotional expression in human-like characters can be processed similarly to human stimuli. However, Kegel et al [15] showed differences in the perception of fearful dynamic avatars and dynamic human faces. Gender [23], culture [8] and race [11] have been reported to influence FER.…”