Virtual friend or threat? The effects of facial expression and gaze interaction on psychophysiological responses and emotional experience

Abstract

The present study aimed to investigate the impact of facial expression, gaze interaction, and gender on attention allocation, physiological arousal, facial muscle responses, and emotional experience in simulated social interactions. Participants viewed animated virtual characters varying in terms of gender, gaze interaction, and facial expression. We recorded facial EMG, fixation duration, pupil size, and subjective experience. Subject's rapid facial reactions (RFRs) differentiated more clearly between the character's happy and angry expression in the condition of mutual eye-to-eye contact. This finding provides evidence for the idea that RFRs are not simply motor responses, but part of an emotional reaction. Eye movement data showed that fixations were longer in response to both angry and neutral faces than to happy faces, thereby suggesting that attention is preferentially allocated to cues indicating potential threat during social interaction.

The observer observed: Frontal EEG asymmetry and autonomic responses differentiate between another person's direct and averted gaze when the face is seen live

“…We admit readily that we have no obvious explanation for these findings and, therefore, in the following, we can provide only some speculation upon this issue. There are previous results showing that, for females, eye contact with opposite-sex avatar faces is not as pleasant as eye contact with same-sex avatars (Schrammel et al, 2009). …”

Learning under your gaze: the mediating role of affective arousal between perceived direct gaze and memory performance

“…However, our male participants did not show superior performance for a male's storytelling overall, but only when the gaze was direct. There is some evidence that motor resonance to facial expressions as measured by facial electromyography is stronger in the context of direct vs. averted gaze (Schrammel, Pannasch, Graupner, Mojzisch, & Velichkovsky, 2009;Soussignan et al, 2013) although this is not a consistent finding (see Mojzisch et al, 2006). In addition, in a brain imaging study, no effect of head and gaze direction was observed on the mirror neuron system activation (Schulte-Rüther, Markowitsch, Fink, & Piefke, 2007).…”

Positive Emotions Broaden and Build

“…Eye contact, studies show, is necessary for facial mimicry to unfold (Schrammel, Pannasch, Graupner, Mojzisch, & Velichkovsky, 2009), and facial mimicry, in turn, is needed to accurately decode what another person is feeling (Maringer, Krumhuber, Fischer, & Niedenthal, 2011). According to Niedenthal and colleagues' Simulation of Smiles (SIMS) model (Niedenthal et al, 2010), brain coupling mediates the effect of facial mimicry on decoding accuracy, whereas eye contact moderates the effect.…”