Audiovisual Integration of Emotional Information from Voice and Face

Kreifelts, Benjamin; Wildgruber, Dirk; Ethofer, Thomas

doi:10.1007/978-1-4614-3585-3_12

Cited by 5 publications

(5 citation statements)

References 106 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In accordance with our expectations, NECT-specific alterations in brain activity were observed in a distributed set of brain regions including face- and voice-selective areas of the visual and auditory cortex, the STS, inferior frontal cortex, insula and thalamus which have all been demonstrated to be involved in the processing of audiovisual non-verbal emotional signals (for reviews see Campanella and Belin, 2007; Brück et al, 2011b; Kreifelts et al, 2013). Moreover, NECT induced altered activation patterns in motor-related regions, the cerebellum and the parietal cortex.…”

Section: Discussionsupporting

confidence: 87%

Non-verbal emotion communication training induces specific changes in brain function and structure

Kreifelts

Jacob

Brück

et al. 2013

Front. Hum. Neurosci.

Self Cite

View full text Add to dashboard Cite

The perception of emotional cues from voice and face is essential for social interaction. However, this process is altered in various psychiatric conditions along with impaired social functioning. Emotion communication trainings have been demonstrated to improve social interaction in healthy individuals and to reduce emotional communication deficits in psychiatric patients. Here, we investigated the impact of a non-verbal emotion communication training (NECT) on cerebral activation and brain structure in a controlled and combined functional magnetic resonance imaging (fMRI) and voxel-based morphometry study. NECT-specific reductions in brain activity occurred in a distributed set of brain regions including face and voice processing regions as well as emotion processing- and motor-related regions presumably reflecting training-induced familiarization with the evaluation of face/voice stimuli. Training-induced changes in non-verbal emotion sensitivity at the behavioral level and the respective cerebral activation patterns were correlated in the face-selective cortical areas in the posterior superior temporal sulcus and fusiform gyrus for valence ratings and in the temporal pole, lateral prefrontal cortex and midbrain/thalamus for the response times. A NECT-induced increase in gray matter (GM) volume was observed in the fusiform face area. Thus, NECT induces both functional and structural plasticity in the face processing system as well as functional plasticity in the emotion perception and evaluation system. We propose that functional alterations are presumably related to changes in sensory tuning in the decoding of emotional expressions. Taken together, these findings highlight that the present experimental design may serve as a valuable tool to investigate the altered behavioral and neuronal processing of emotional cues in psychiatric disorders as well as the impact of therapeutic interventions on brain function and structure.

show abstract

Section: Discussionsupporting

confidence: 87%

Non-verbal emotion communication training induces specific changes in brain function and structure

Kreifelts

Jacob

Brück

et al. 2013

Front. Hum. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The pSTS has been demonstrated to be involved in the integration of audiovisual information for a broad variety of stimuli [Beauchamp et al, 2004;Calvert et al, 2000;van Atteveldt et al, 2004;] including affective [Kreifelts et al, 2013;Watson et al, 2014b] and identity information [Joassin et al, 2011;Watson et al, 2014a] from faces and voices. The pSTS has been demonstrated to be involved in the integration of audiovisual information for a broad variety of stimuli [Beauchamp et al, 2004;Calvert et al, 2000;van Atteveldt et al, 2004;] including affective [Kreifelts et al, 2013;Watson et al, 2014b] and identity information [Joassin et al, 2011;Watson et al, 2014a] from faces and voices.…”

Section: Figurementioning

confidence: 99%

“…Our results show an effect of face primes on voice identity processing in the pSTS and in the angular gyrus, which have both been shown to respond to different modalities in various tasks. The pSTS has been demonstrated to be involved in the integration of audiovisual information for a broad variety of stimuli [Beauchamp et al, 2004;Calvert et al, 2000;van Atteveldt et al, 2004;] including affective [Kreifelts et al, 2013;Watson et al, 2014b] and identity information [Joassin et al, 2011;Watson et al, 2014a] from faces and voices. Animal research has shown that the STS is directly connected with auditory and visual cortices [Seltzer and Pandya, 1994]; and comprises multisensory neurons which respond to both, auditory and visual stimulation [Barraclough et al, 2005;Perrodin et al, 2014].…”

Section: Figurementioning

confidence: 99%

Activation in the angular gyrus and in the pSTS is modulated by face primes during voice recognition

Hölig

Föcker

Best

et al. 2017

Human Brain Mapping

View full text Add to dashboard Cite

The aim of the present study was to better understand the interaction of face and voice processing when identifying people. In a S1-S2 crossmodal priming fMRI experiment, the target (S2) was a disyllabic voice stimulus, whereas the modality of the prime (S1) was manipulated blockwise and consisted of the silent video of a speaking face in the crossmodal condition or of a voice stimulus in the unimodal condition. Primes and targets were from the same speaker (person-congruent) or from two different speakers (person-incongruent). Participants had to classify the S2 as either an old or a young person. Response times were shorter after a congruent than after an incongruent face prime. The right posterior superior temporal sulcus (pSTS) and the right angular gyrus showed a significant person identity effect (person-incongruent > person-congruent) in the crossmodal condition but not in the unimodal condition. In the unimodal condition, a person identity effect was observed in the bilateral inferior frontal gyrus. Our data suggest that both the priming with a voice and with a face result in a preactivated voice representation of the respective person, which eventually facilitates (person-congruent trials) or hampers (person-incongruent trials) the processing of the identity of a subsequent voice. This process involves activation in the right pSTS and in the right angular gyrus for voices primed by faces, but not for voices primed by voices. Hum Brain Mapp 38:2553-2565, 2017. © 2017 Wiley Periodicals, Inc.

show abstract

“…With experience and the maturation of sensory and perceptual systems, this eventually develops into the ability to recognize and discriminate emotions. This recognition has exhibited a robust multisensory effect that seems to be automatic (de Gelder, Stienen, & Van den Stock, 2013;Kreifelts, Wildgruber, & Ethofer, 2013;Pourois & Dhar, 2013) and apparently occurs at the early stage after stimulus onset, revealing an early perceptual (Pourois & Dhar, 2013) or a later cognitive process (de Gelder et al, 2013). Not all emotions are perceived with equal ease (Abelin & Allwood, 2000;McAllister & Hansson, 1995;Most, 1994), and their recognition may be affected by various factors, such as context (e.g.…”

Section: Introductionmentioning

confidence: 97%

Recognition and production of emotions in children with cochlear implants

Mildner

Koska

2014

Clinical Linguistics & Phonetics

View full text Add to dashboard Cite

The aim of this study was to examine auditory recognition and vocal production of emotions in three prelingually bilaterally profoundly deaf children aged 6-7 who received cochlear implants before age 2, and compare them with age-matched normally hearing children. No consistent advantage was found for the normally hearing participants. In both groups, sadness was recognized best and disgust was the most difficult. Confusion matrices among other emotions (anger, happiness, and fear) showed that children with and without hearing impairment may rely on different cues. Both groups of children showed that perception is superior to production. Normally hearing children were more successful in the production of sadness, happiness, and fear, but not anger or disgust. The data set is too small to draw any definite conclusions, but it seems that a combination of early implantation and regular auditory-oral-based therapy enables children with cochlear implants to process and produce emotional content comparable with children with normal hearing.

show abstract

Audiovisual Integration of Emotional Information from Voice and Face

Cited by 5 publications

References 106 publications

Non-verbal emotion communication training induces specific changes in brain function and structure

Non-verbal emotion communication training induces specific changes in brain function and structure

Activation in the angular gyrus and in the pSTS is modulated by face primes during voice recognition

Recognition and production of emotions in children with cochlear implants

Contact Info

Product

Resources

About