The Effects of Dynamic and Static Emotional Facial Expressions of Humans and Their Avatars on the EEG: An ERP and ERD/ERS Study

Abstract

This study aimed to examine whether the cortical processing of emotional faces is modulated by the computerization of face stimuli (”avatars”) in a group of 25 healthy participants. Subjects were passively viewing 128 static and dynamic facial expressions of female and male actors and their respective avatars in neutral or fearful conditions. Event-related potentials (ERPs), as well as alpha and theta event-related synchronization and desynchronization (ERD/ERS), were derived from the EEG that was recorded during the task. All ERP features, except for the very early N100, differed in their response to avatar and actor faces. Whereas the N170 showed differences only for the neutral avatar condition, later potentials (N300 and LPP) differed in both emotional conditions (neutral and fear) and the presented agents (actor and avatar). In addition, we found that the avatar faces elicited significantly stronger reactions than the actor face for theta and alpha oscillations. Especially theta EEG frequencies responded specifically to visual emotional stimulation and were revealed to be sensitive to the emotional content of the face, whereas alpha frequency was modulated by all the stimulus types. We can conclude that the computerized avatar faces affect both, ERP components and ERD/ERS and evoke neural effects that are different from the ones elicited by real faces. This was true, although the avatars were replicas of the human faces and contained similar characteristics in their expression.

Digital Health to Patient-Facing Artificial Intelligence: Ethical Implications and Threats to Dignity for Patients With Cancer

“…Dynamic avatars with human-like characteristics are more persuasive and better received by patients than static avatars. 17 To date, this has been accomplished through hybrid human-supervised AI relational agents (so-called human-in-the-loop), fostering more natural and engaging communication, which can promote positive changes such as reducing loneliness and anxiety. 18 There is great potential for these technologies to provide supportive care to patients with cancer.…”

Electrophysiological correlates of face and object perception: A comparative analysis of 2D laboratory and virtual reality conditions

“…Representing realistic and, at the same time, dynamic and responsive human avatars in virtual environments remains a demanding task (Di Natale et al, 2023;Hepperle et al, 2022). Virtual faces created with today's advanced digital graphics technologies, although appearing remarkably realistic, are still perceived as not fully human, which is reflected at both the behavioral and neural levels (uncanny valley effect; Di Natale et al, 2023;Hepperle et al, 2022;Miller et al, et al, 2017;Sollfrank et al, developments also have an impact on practical research areas that focus on recognizing, interpreting, and responding to human emotions, such as emotional artificial intelligence (AI), where AI systems are used to improve the interaction between machines and humans in this regard. Here, EEG data is being utilized to gain deeper insights into emotion recognition processes, with the aim of modeling them (Kamble & Sengupta, 2023;J.…”

“…Furthermore, there is increasing recognition that complex processes like face perception cannot be fully captured by early exogenous ERPs. The N170 primarily captures stimulus‐specific information (Berchicci et al., 2016), whereas later components such as the early posterior negativity (EPN: ≈200–400 ms at posterior electrodes) or the late positive potential (LPP: ≈400–1000 ms at contro‐parietal electrodes) (e.g., Bublatzky et al., 2014; Herbert et al., 2013; Schindler et al., 2017; Wheatley et al., 2011) provide insights into more comprehensive stimulus processing (Nasr, 2010; Ratcliff et al., 2009; Zion‐Golumbic & Bentin, 2007), influenced by social relevance, emotions, and decision‐relevant factors (Bublatzky et al., 2014; Herbert et al., 2013; Sollfrank et al., 2021; Stolz et al., 2019). These insights can be further strengthened by locking the ERPs to participant responses (Nasr, 2010), as later components related to the processing of perceptual decisions are more strongly associated with the response than the stimulus onset (Berchicci et al., 2016).…”

A Smart Mirror for Emotion Monitoring in Home Environments

“…The list of face attributes returned by the face attribute estimation module together with the emotion recognized by the face emotion recognition module is used by the Avatar module based on the Avatars library [67] to create an avatar that represents the current user. The choice of an avatar to represent the user's face makes the system more attractive [68]. Furthermore, avatars were demonstrated to be very effective for therapeutic purposes [69,70].…”