Statistical pattern recognition reveals shared neural signatures for displaying and recognizing specific facial expressions

Abstract

Human neuroimaging and behavioural studies suggest that somatomotor ‘mirroring’ of seen facial expressions may support their recognition. Here we show that viewing specific facial expressions triggers the representation corresponding to that expression in the observer’s brain. Twelve healthy female volunteers underwent two separate fMRI sessions: one where they observed and another where they displayed three types of facial expressions (joy, anger and disgust). Pattern classifier based on Bayesian logistic regression was trained to classify facial expressions (i) within modality (trained and tested with data recorded while observing or displaying expressions) and (ii) between modalities (trained with data recorded while displaying expressions and tested with data recorded while observing the expressions). Cross-modal classification was performed in two ways: with and without functional realignment of the data across observing/displaying conditions. All expressions could be accurately classified within and also across modalities. Brain regions contributing most to cross-modal classification accuracy included primary motor and somatosensory cortices. Functional realignment led to only minor increases in cross-modal classification accuracy for most of the examined ROIs. Substantial improvement was observed in the occipito-ventral components of the core system for facial expression recognition. Altogether these results support the embodied emotion recognition model and show that expression-specific somatomotor neural signatures could support facial expression recognition.

Feelings first? Sex differences in affective and cognitive processes in emotion recognition

“…The accurate and fast recognition of emotions from others' facial expressions is important for effective social interaction and communication (Blair, 2003). It has been proposed that emotions expressed through the face are recognised through a process of embodied simulation (Gallese, 2005), whereby others' expressions activate corresponding sensorimotor representations in the observer's brain, and this is supported by recent neuroimaging evidence (Volynets et al, 2020). The embodied response may involve mimicry, which is the subthreshold activation of facial muscles involved in producing the target expression, as demonstrated in electromyography (EMG) studies (Sato et al, 2008).…”

Naturalistic Stimuli in Affective Neuroimaging: A Review

“…Portrayed emotions have been extracted mainly with manual annotations (Ekman and Oster, 1979;Lahnakoski et al, 2012;Witkower and Tracy, 2019). Emotion cues used in labeling portrayed emotions include, for instance, facial behaviors (e.g., frowning, Volynets et al, 2020), bodily behaviors (e.g., Witkower and Tracy, 2019), verbal and vocal cues (e.g., trembling voice, Ethofer et al, 2009), actions, and contextual cues (Skerry and Saxe, 2014). Perception of emotions operates at an abstract level regardless of the stimulus modality: both inferred (from contextual descriptions) and perceived (from facial expressions) have shared neural codes (Skerry and Saxe, 2014).…”

“…However, it is possible that the brain activity patterns measured with affective stimuli reflect also automatic emotional behaviors and inhibition, as suggested by the automatic facial behaviors evoked during emotional movie viewing (Chang et al, 2021). For instance, displaying and observing facial behaviors activate partly overlapping brain regions (Volynets et al, 2020). Similarly, observing actions in movies and real-life automatically activates overlapping motor regions (Nummenmaa et al, 2014c;Smirnov et al, 2017).…”

Functional organization of social perception in the human brain

“…Previous classification studies have addressed the importance of these regions for distinct social perceptual tasks. For example, faces, animals and objects have been successfully classified by respective brain activation patterns in areas within FG and LOTC 20 , and highest classification accuracy for facial expressions is typically observed in FG, STS and anterior temporal cortex 22,62,63 . Additionally, goal-oriented actions can be reliably classified in LOTC and interior parietal lobe 25,64 .…”