Incongruence Between Observers’ and Observed Facial Muscle Activation Reduces Recognition of Emotional Facial Expressions From Video Stimuli

Abstract

According to embodied cognition accounts, viewing others’ facial emotion can elicit the respective emotion representation in observers which entails simulations of sensory, motor, and contextual experiences. In line with that, published research found viewing others’ facial emotion to elicit automatic matched facial muscle activation, which was further found to facilitate emotion recognition. Perhaps making congruent facial muscle activity explicit produces an even greater recognition advantage. If there is conflicting sensory information, i.e., incongruent facial muscle activity, this might impede recognition. The effects of actively manipulating facial muscle activity on facial emotion recognition from videos were investigated across three experimental conditions: (a) explicit imitation of viewed facial emotional expressions (stimulus-congruent condition), (b) pen-holding with the lips (stimulus-incongruent condition), and (c) passive viewing (control condition). It was hypothesised that (1) experimental condition (a) and (b) result in greater facial muscle activity than (c), (2) experimental condition (a) increases emotion recognition accuracy from others’ faces compared to (c), (3) experimental condition (b) lowers recognition accuracy for expressions with a salient facial feature in the lower, but not the upper face area, compared to (c). Participants (42 males, 42 females) underwent a facial emotion recognition experiment (ADFES-BIV) while electromyography (EMG) was recorded from five facial muscle sites. The experimental conditions’ order was counter-balanced. Pen-holding caused stimulus-incongruent facial muscle activity for expressions with facial feature saliency in the lower face region, which reduced recognition of lower face region emotions. Explicit imitation caused stimulus-congruent facial muscle activity without modulating recognition. Methodological implications are discussed.

A False Trail to Follow: Differential Effects of the Facial Feedback Signals From the Upper and Lower Face on the Recognition of Micro-Expressions

“…In addition, the more voluntary movement in the lower face means that it relies more on attention and consciousness. Such a process may result in the inconsistent or inaccurate mimicking in the lower face when the expression is very fast, subtle, or ambiguous ( Wood et al, 2016b ; Wu et al, 2016 ; Wingenbach et al, 2018 ). Future studies should address these possible underlying mechanisms.…”

“…By using such a design, we did not find any significant moderation effects of emotion category on facial feedback across our three studies. However, the embodied cognition accounts postulate that, during the process of emotion perception, specific facial muscle manipulation should have its specific effect on specific category of facial expressions (e.g., Oberman et al, 2007 ; Ponari et al, 2012 ; Wingenbach et al, 2018 ). This question should be further investigated by employing more sufficient trial numbers in the future.…”

“…For example, the picture of a smiling mouth was found to bias perceived expression in upper face toward happiness, even when the actual upper face was not happy ( Kontsevich and Tyler, 2004 ; Calvo et al, 2013a , b ). By employing electromyography (EMG), one recent study on facial feedback also provides the indirect evidence to suggest that, during explicit facial imitation of macro-expressions, facial muscles on the lower face (e.g., depressor) might be more activated than the upper facial muscles (e.g., corrugator, frontalis; Wingenbach et al, 2018 ). Additional evidence suggests that the lower face also has more weight in the perception of micro-expressions.…”

The intensity of emotion: Altered motor simulation impairs processing of facial expressions in congenital facial palsy

“…pens or chopsticks [Ponari, Conson, D'Amico, Grossi, & Trojano, 2012], mouthguards [Rychlowska et al, 2014], hardening gels , botulinum [Baumeister, Papa, & Foroni, 2016]) and in case of individuals suffering from congenital or acquired facial and expression production disorders, difficulties in recognizing facial expressions have been reported (Bate, Cook, Mole, & Cole, 2013;Calder, Keane, Cole, Campbell, & Young, 2000;Giannini et al, 1984;McKone & Robbins, 2011;Nicolini et al, 2019;De Stefani et al 2019). This whole body of evidence supports the notion that the simulation mechanism may optimally work when facial feedback is consistent with the internal simulation (Wolpert & Flanagan, 2001); thus, interfering with facial mimicry may cause an incongruent signal with the pattern of sensorimotor activity and would result in a drop of recognition/discrimination accuracy (Wingenbach, Brosnan, Pfaltz, Plichta, & Ashwin, 2018;.…”

Naturalistic Emotion Decoding From Facial Action Sets

“…Furthermore, although we lacked data regarding emotions experienced by the expressers and we did not monitor the internal states of the participants, it would have been interesting to investigate interactions between AUs and encoded/decoded emotions and the characteristics of the observers. Given the literature on how emotions, facial mimicry and moods of observers influence emotion perception in others (e.g., Schmid and Schmid Mast, 2010; Wood et al, 2016; Wingenbach et al, 2018) it is a valuable topic in future research on dynamic naturalistic stimuli interpretation. Second, we analyzed only male participants.…”