Face-selective regions differ in their ability to classify facial expressions

Abstract

Recognition of facial expressions is crucial for effective social interactions. Yet, the extent to which the various face-selective regions in the human brain classify different facial expressions remains unclear. We used functional magnetic resonance imaging (fMRI) and support vector machine pattern classification analysis to determine how well face-selective brain regions are able to decode different categories of facial expression. Subjects participated in a slow event-related fMRI experiment in which they were shown 32 face pictures, portraying four different expressions: neutral, fearful, angry, and happy and belonging to eight different identities. Our results showed that only the amygdala and the posterior superior temporal sulcus (STS) were able to accurately discriminate between these expressions, albeit in different ways: The amygdala discriminated fearful faces from non-fearful faces, whereas STS discriminated neutral from emotional (fearful, angry and happy) faces. In contrast to these findings on the classification of emotional expression, only the fusiform face area (FFA) and anterior inferior temporal cortex (aIT) could discriminate among the various facial identities. Further, the amygdala and STS were better than FFA and aIT at classifying expression, while FFA and aIT were better than the amygdala and STS at classifying identity. Taken together, our findings indicate that the decoding of facial emotion and facial identity occurs in different neural substrates: the amygdala and STS for the former and FFA and aIT for the latter.

Spatiotemporal dynamics in human visual cortex rapidly encode the emotional content of faces

“…Our results are in line with previous fMRI MVPA studies demonstrating above‐chance expression decoding in all face‐selective regions (Wegrzyn et al, 2015) and particularly in the FFA, STS, and amygdala, in the absence of univariate effects (Zhang et al, 2016a). Notably, the latter found that the STS could classify neutral and emotional faces above chance, whereas here we show an advantage for angry expressions.…”

“…To move beyond the limitations of sensor‐space spatial inference in our MVPA analysis (including concerns of signal leakage, head motion and inter‐individual variability; Zhang et al, 2016), the data were projected into source space using the linearly constrained minimum variance (LCMV) beamformer (Hillebrand et al, 2005; Van Veen, van Drongelen, Yuchtman, & Suzuki, 1997). This approach combines the forward model and the data covariance matrix to construct an adaptive spatial filter.…”

“…Previous studies have demonstrated successful decoding of facial expression from face‐selective areas using fMRI (Wegrzyn et al, 2015; Zhang et al, 2016a). Some studies have also decoded emotion from EEG data (Kashihara, 2014; Li et al, 2009; Petrantonakis, & Hadjileontiadis, 2010) and target happy expressions from MEG data (Cecotti et al, 2017), while an intracranial EEG study has demonstrated late decoding of facial expression (fear and happiness) from the human fusiform gyrus (Tsuchiya et al, 2008).…”

Spatio-temporal dynamics of face perception

“…One main feature in these models is the separation of the processing of changeable and invariant aspects. Changeable aspects, (or motion: Bernstein & Yovel, 2015) such as expressions, are mainly processed in the dorsal stream, especially in the superior temporal sulcus (STS; Greening, Mitchell, & Smith, 2018;Said, Haxby, & Todorov, 2011;Zhang et al, 2016). In contrast, invariant aspects, such as identity, are processed in the ventral stream from part-based processing in the occipital face area (OFA; Atkinson & Adolphs, 2011;Henriksson, Mur, & Kriegeskorte, 2015;Pitcher, Walsh, & Duchaine, 2011), to the fusiform face area (FFA; Anzellotti, Fairhall, & Caramazza, 2014;Carlin & Kriegeskorte, 2017;Dobs, Schultz, Bülthoff, & Gardner, 2018;Kanwisher & Yovel, 2006), and finally to highestlevel, viewpoint-invariant processing in the ventral anterior temporal lobe (vATL; Anzellotti & Caramazza, 2016;Anzellotti et al, 2014;Collins & Olson, 2015;Kriegeskorte, Formisano, Sorger, & Goebel, 2007).…”

Violence and Latin‐American preadolescents: A study of social brain function and cortisol levels

“…Recently, a combination of machine‐learning algorithms and fMRI data was used to decode face‐selective regions. The study showed that only the amygdala and the posterior superior temporal sulcus (STS) accurately discriminated between neutral faces and emotional faces (Zhang, Japee, Nolan, Chu, Liu, & Ungerleider, ).…”