Cortical responses to dynamic emotional facial expressions generalize across stimuli, and are sensitive to task-relevance, in adults with and without Autism

Kliemann, Dorit; Richardson, Hilary; Anzellotti, Stefano; Ayyash, Dima; Haskins, Amanda J.; Gabrieli, John D. E.; Saxe, Rebecca

doi:10.1016/j.cortex.2018.02.006

Cited by 29 publications

(29 citation statements)

References 70 publications

(91 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent evidence, however, indicates that recognition of face identity and facial expressions might be more integrated than previously thought. The valence of facial expressions can be decoded from ventral temporal regions including OFA and FFA (45,28). At the same time, identity information can be decoded from the face-selective pSTS (2,18,11).…”

Section: Challenges To the Dominant Viewmentioning

confidence: 99%

Recognition of identity and expressions as integrated processes

O'Nell¹,

Saxe²,

Anzellotti³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

According to the dominant account of face processing, recognition of emotional expressions is implemented by the superior temporal sulcus (STS), while recognition of face identity is implemented by inferior temporal cortex (IT) (Haxby et al., 2000). However, recent patient and imaging studies (Fox et al., 2011, Anzellotti et al. 2017) found that the STS also encodes information about identity.Jointly representing expression and identity might be computationally advantageous: learning to recognize expressions could lead to the emergence of representations that support identity recognition. To test this hypothesis, we trained a deep densely connected convolutional network (DenseNet, Huang et al., 2017) to classify face images from the fer2013 dataset as either angry, disgusted, afraid, happy, sad, surprised, or neutral. We then froze the weights of the DenseNet and trained linear layers attached to progressively deeper layers of this net to classify either emotion or identity using a subset of the Karolinska (KDEF) dataset. Finally, we tested emotion and identity classification in left out images in the KDEF dataset that were not used for training.Classification accuracy for emotions in the KDEF dataset increased from early to late layers of the DenseNet, indicating successful transfer across datasets. Critically, classification accuracy for identity also increased from early to late layers of this DenseNet, despite the fact that it had not been trained to classify identity. A linear layer trained on the DenseNet features vastly outperformed a linear layer trained on pixels (98.8% vs 68.7%), demonstrating that the high accuracy obtained with the DenseNet features cannot be explained by low-level confounds. These results show that learning to recognize facial expressions can lead to the spontaneous emergence of representations that support the recognition of identity, thus offering a principled computational account for the discovery of expression and identity representations within the same portion of STS.

show abstract

Section: Challenges To the Dominant Viewmentioning

confidence: 99%

Recognition of identity and expressions as integrated processes

O'Nell¹,

Saxe²,

Anzellotti³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Such difficulties have been reported both for facial emotion (Sato, Toichi, Uono, & Kochiyama, 2012;Sato, Uono, & Toichi, 2013), and visual-speech information (Foxe et al, 2015;Schelinski, Riedel, & von Kriegstein, 2014). These deficits are associated with reduced brain responses in ASD in visual sensory cortices (i.e., V5/MT, pSTS/STG [TVSA] and FFA; Borowiak et al, 2018;Pelphrey, Morris, McCarthy, & LaBar, 2007;Sato et al, 2012; but see Kliemann et al, 2018), and also with decreased functional connectivity between the two dorsal-movement pathway regions (Borowiak et al, 2018). Given the importance of visual-speech perception for successful face-to-face communication (Arnold & Hill, 2001;Ross et al, 2007;Sumby & Pollack, 1954), less efficient processing of visual speech may contribute to speech comprehension difficulties in face-to-face situations in ASD (Smith and Bennetto, 2007;Schelinski et al, 2014).…”

mentioning

confidence: 98%

Dorsal‐movement and ventral‐form regions are functionally connected during visual‐speech recognition

Borowiak

Maguinness

Kriegstein

2019

Human Brain Mapping

View full text Add to dashboard Cite

Faces convey social information such as emotion and speech. Facial emotion processing is supported via interactions between dorsal-movement and ventral-form visual cortex regions. Here, we explored, for the first time, whether similar dorsalventral interactions (assessed via functional connectivity), might also exist for visualspeech processing. We then examined whether altered dorsal-ventral connectivity is observed in adults with high-functioning autism spectrum disorder (ASD), a disorder associated with impaired visual-speech recognition. We acquired functional magnetic resonance imaging (fMRI) data with concurrent eye tracking in pairwise matched control and ASD participants. In both groups, dorsal-movement regions in the visual motion area 5 (V5/MT) and the temporal visual speech area (TVSA) were functionally connected to ventral-form regions (i.e., the occipital face area [OFA] and the fusiform face area [FFA]) during the recognition of visual speech, in contrast to the recognition of face identity. Notably, parts of this functional connectivity were decreased in the ASD group compared to the controls (i.e., right V5/MT-right OFA, left TVSA-left FFA). The results confirmed our hypothesis that functional connectivity between dorsal-movement and ventral-form regions exists during visual-speech processing. Its partial dysfunction in ASD might contribute to difficulties in the recognition of dynamic face information relevant for successful face-to-face communication. K E Y W O R D S atypical perception, dynamic face perception, fMRI, form, functional connectivity, highfunctioning ASD, lip reading, movement

show abstract

“…Likewise, at a neural level, individuals with ASD showed attenuated brain activity in the TPJ during various socio-cognitive tasks targeting ToM [44]- [46] (but see [47], [48]). On the other hand, it has been gradually recognized that many individuals with ASD, especially the higher functioning and verbally gifted ones, are able to pass these ToM tasks by means of compensatory sensory strategies and rule-based reasoning [49] despite substantial impairments in spontaneous social communication and interaction in daily life.…”

Section: Introductionmentioning

confidence: 99%

Social touch observation in adults with autism: intact neural representations of affective meaning but lack of embodied resonance

Pillet

Amelynck

et al. 2019

Preprint

View full text Add to dashboard Cite

Humans can easily grasp the affective meaning of touch when observing social interactions.Several neural systems support this ability, including theory of mind (ToM) and somatosensory resonance systems, but it is unclear how these systems are affected in autism spectrum disorder (ASD). Individuals with ASD are characterized by impairments in social interaction and the use of (non)verbal communication such as social and reciprocal touch. The present study applies an ecologically valid stimulus set and multivoxel pattern fMRI neuroimaging to pinpoint atypicalities in the neural circuitry underlying socio-affective touch observation in adults with ASD as compared to matched neurotypical controls. The MVPA results reveal that the affective meaning of touch is well represented in the temporoparietal junction, a core ToM mentalizing area, in both groups. Conversely, only the neurotypical group hosts affective touch representations in the somatosensory cortex, not the ASD group, yielding a significant group difference. Lastly, individuals with a more positive attitude towards receiving, witnessing, and providing social touch and with a higher score on social responsivity, show more differentiated representations of the affective meaning of touch in these somatosensory areas. Together, our findings imply that individuals with ASD are able to cognitively represent the affective meaning of touch, but they lack the spontaneous embodied somatosensory resonance when observing social touch communications. Individual differences in this diminished resonance appear to be related to social touch avoidance and quantitative autism traits. Significance StatementAutism is characterized by socio-communicative impairments, including abnormal processing of interpersonal touch. Little is known about the neural basis of atypicalities in social touch processing in autism. Here, adults with and without autism watched video clips displaying social touch interactions and judged the affective valence of the touch. Subsequently, they underwent functional magnetic resonance imaging while watching the same videos. Brain activity patterns demonstrate that adults with autism show intact cognitive understanding (i.e. "knowing") of observed socio-affective touch experiences but lack of embodied emotional resonance (i.e. "feeling"). This lack of emotional resonance is linked to social touch avoidance and quantitative autism traits. These findings highlight that the depth of experiencing the state of others is shallower in people with autism.

show abstract

Cortical responses to dynamic emotional facial expressions generalize across stimuli, and are sensitive to task-relevance, in adults with and without Autism

Cited by 29 publications

References 70 publications

Recognition of identity and expressions as integrated processes

Recognition of identity and expressions as integrated processes

Dorsal‐movement and ventral‐form regions are functionally connected during visual‐speech recognition

Social touch observation in adults with autism: intact neural representations of affective meaning but lack of embodied resonance

Contact Info

Product

Resources

About