Decoding facial expressions based on face‐selective and motion‐sensitive areas

Abstract

Humans can easily recognize others' facial expressions. Among the brain substrates that enable this ability, considerable attention has been paid to face-selective areas; in contrast, whether motion-sensitive areas, which clearly exhibit sensitivity to facial movements, are involved in facial expression recognition remained unclear. The present functional magnetic resonance imaging (fMRI) study used multi-voxel pattern analysis (MVPA) to explore facial expression decoding in both face-selective and motion-sensitive areas. In a block design experiment, participants viewed facial expressions of six basic emotions (anger, disgust, fear, joy, sadness, and surprise) in images, videos, and eyes-obscured videos. Due to the use of multiple stimulus types, the impacts of facial motion and eye-related information on facial expression decoding were also examined. It was found that motion-sensitive areas showed significant responses to emotional expressions and that dynamic expressions could be successfully decoded in both face-selective and motion-sensitive areas. Compared with static stimuli, dynamic expressions elicited consistently higher neural responses and decoding performance in all regions. A significant decrease in both activation and decoding accuracy due to the absence of eye-related information was also observed. Overall, the findings showed that emotional expressions are represented in motion-sensitive areas in addition to conventional face-selective areas, suggesting that motion-sensitive regions may also effectively contribute to facial expression recognition. The results also suggested that facial motion and eye-related information played important roles by carrying considerable expression information that could facilitate facial expression recognition. Hum Brain Mapp 38:3113-3125, 2017. © 2017 Wiley Periodicals, Inc.

Statistical Pattern Recognition Reveals Shared Neural Signatures for Displaying and Recognizing Specific Facial Expressions

“…Furthermore, both damage to somatosensory cortex (Adolphs et al 2000) and its inactivation by transcranial magnetic stimulation (TMS; Pourtois et al 2004) impairs recognizing emotions from facial expressions, suggesting that somatomotor embodiment of seen emotions supports their recognition. In line with these results, emotional facial expressions can be successfully decoded from motor brain regions (Liang et al, 2017). Yet strong support for the embodied recognition view would require showing that i) both displaying and seeing different facial expressions would trigger expression-specific , discrete neural signatures in the somatomotor system and that ii) these expression-specific neural signatures would be corresponding when displaying and observing the expressions.…”

“…Said et al 2010; Peelen et al 2010; Harry et al 2013; Wegrzyn et al 2015), we also confirmed that viewing the facial expressions was associated with distinct expression-specific activation patterns, particularly in the fusiform and inferior occipital cortices, V5, and STS. However, seen expressions could also be successfully decoded from regional activation patterns in the somatosensory (see also Kragel & LaBar, 2016) and motor cortices (see also Liang et al, 2017), and components of the emotion circuit (amygdala, ACC), suggesting that expression-specific affective and somatomotor codes are also activated during facial expression perception.…”

Abstract Representations of Emotions Perceived From the Face, Body, and Whole-Person Expressions in the Left Postcentral Gyrus

“…Functional images were acquired using a 3.0 T Siemens scanner in Yantai Hospital Affiliated to Binzhou Medical University with a twenty-channel head coil. Foam pads and earplugs were used to reduce the head motion and scanner noise ( Liang et al, 2017 ). For functional scans, an echo-planar imaging (EPI) sequence was used (T2 ∗ weighted, gradient echo sequence), with the following parameters: TR (repetition time) = 2000 ms, TE (echo time) = 30 ms, voxel size = 3.1 mm × 3.1 mm × 4.0 mm, matrix size = 64 × 64, 33 axial slices, 0.6 mm slices gap, FA = 90°.…”

Linear Representation of Emotions in Whole Persons by Combining Facial and Bodily Expressions in the Extrastriate Body Area

“…Behavioral studies have shown that the intact bodies can be visually perceived better than the body parts ( Soria Bauser and Suchan, 2013 ). However, the use of static and neutral images in previous studies has limited the interpretation of the data ( Liang et al, 2017 ). Thus, it remains unclear how the combination of faces and bodies is influenced by dynamic emotion information, which may activate just one specific network.…”

“…However, the coactivated patterns for multiple voxels can now be examined with the development of fMRI data analysis approaches. As compared with the traditional measure of the mean response magnitude, richer information on neural representations can be provided by the voxel-by-voxel activation patterns, and at a finer scale ( Haynes and Rees, 2006 ; Norman et al, 2006 ; Liang et al, 2017 ). The two scenarios suggest different predictions for the pattern associations.…”