Similar and distinct neural mechanisms of visual and auditory emotion perception

Zhang, Yao; Zhang, Fan; Bi, Taiyong; Qiu, Jiang

doi:10.1360/n972018-00721

Cited by 4 publications

(6 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous studies have shown that the bilateral STG were significantly activated when hearing emotional sounds compared to neutral sounds (Ethofer et al 2012), and that the right STG was associated with extracting tonal cues for emotional inferences, while the bilateral STG was involved in processing sonic cues of emotional prosody (Dara 2012). Studies also showed that the right STG played an important role in emotional facial perception of different valences (Zhang et al 2019). Furthermore, the STG was also revealed to participate in processing of emotion perception in audio-visual modality.…”

Section: The Perception Of Speech Emotion In Audio-visual Modalitymentioning

confidence: 93%

“…The STG was often revealed to be involved in the unimodal emotion perception (Dara 2012;Ethofer et al 2012;Zhang et al 2019). Previous studies have shown that the bilateral STG were significantly activated when hearing emotional sounds compared to neutral sounds (Ethofer et al 2012), and that the right STG was associated with extracting tonal cues for emotional inferences, while the bilateral STG was involved in processing sonic cues of emotional prosody (Dara 2012).…”

Section: The Perception Of Speech Emotion In Audio-visual Modalitymentioning

confidence: 99%

“…A recent study using metaanalysis of 18 neuroimaging studies identified a core audio-visual emotion processing network, including the right posterior superior temporal gyrus, the left anterior superior temporal gyrus, the right amygdala and the thalamus (Gao et al 2019). The important role of the right superior temporal gyrus in facial perception of different emotion valence was further reinforced by examining the neural mechanism of emotion perception in visual and auditory modalities via the multivoxel pattern analysis (Zhang et al 2019). These studies have expanded our understanding of emotional speech perception.…”

Section: Introductionmentioning

confidence: 99%

“…Numerous studies have revealed the role of the superior temporal gyrus in multimodal affective perception and integration (Robins et al 2009;Leitman et al 2010;Park et al 2010;Jeong et al 2011;Müller et al 2012;Hagan et al 2013;Jessen and Kotz 2015;Gao et al 2019;Zhang et al 2019). The insula has been pointed out to be involved in the perception and experience of emotions (Duerden et al 2013), and the insula cortex have been confirmed to be necessary and sufficient platform for human emotions , which was actually the only neural source of emotion experience (Damasio et al 2013).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Weighted RSA: An Improved Framework on the Perception of Audio-visual Affective Speech in Left Insula and Superior Temporal Gyrus

Dong

et al. 2021

Neuroscience

View full text Add to dashboard Cite

Section: The Perception Of Speech Emotion In Audio-visual Modalitymentioning

confidence: 93%

Section: The Perception Of Speech Emotion In Audio-visual Modalitymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Weighted RSA: An Improved Framework on the Perception of Audio-visual Affective Speech in Left Insula and Superior Temporal Gyrus

Dong

et al. 2021

Neuroscience

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

Weighted RSA: an improved framework on the perception of audio-visual affective speech in left insula and superior temporal gyrus

Dong

Guo

et al. 2020

Preprint

View full text Add to dashboard Cite

Being able to accurately perceive the emotion expressed by the facial or verbal expression from others is critical to successful social interaction. However, only few studies examined the multimodal interactions on speech emotion, and there is no consistence in studies on the speech emotion perception. It remains unclear, how the speech emotion of different valence is perceived on the multimodal stimuli by our human brain. In this paper, we conducted a functional magnetic resonance imaging (fMRI) study with an event-related design, using dynamic facial expressions and emotional speech stimuli to express different emotions, in order to explore the perception mechanism of speech emotion in audio-visual modality. The representational similarity analysis (RSA), whole-brain searchlight analysis, and conjunction analysis of emotion were used to interpret the representation of speech emotion in different aspects. Significantly, a weighted RSA approach was creatively proposed to evaluate the contribution of each candidate model to the best fitted model. The results of weighted RSA indicated that the fitted models were superior to all candidate models and the weights could be used to explain the representation of ROIs. The bilateral amygdala has been shown to be associated with the processing of both positive and negative emotions except neutral emotion. It is indicated that the left posterior insula and the left anterior superior temporal gyrus (STG) play important roles in the perception of multimodal speech emotion.

show abstract

Effect of Interaction Effects Between Visual and Auditory Signs on Consumer Purchasing Behavior Based on AISAS Model

Li¹,

Younghwan²

2023

Preprint

View full text Add to dashboard Cite

This study, based on the AISAS model, explores the impact of the interaction effect between visual and auditory signals on consumer purchase behavior. Using experimental methods, 120 participants were randomly assigned to four different visual and auditory signal combinations, and their purchase intentions and actual purchase behavior were measured. The results show that the interaction effect between visual and auditory signals has a significant impact on both purchase intentions and actual purchase behavior, and there is a significant positive relationship. Specifically, when visual and auditory signals are mutually consistent, consumers have the highest purchase intentions and actual purchase behavior; when both visual and auditory signals are absent, consumers have the lowest purchase intentions and actual purchase behavior; when either the visual or auditory signal is missing, consumers' purchase intentions and actual purchase behavior are in between the two extremes. This study provides a new perspective for understanding consumers' decision-making processes in multi-sensory environments and offers valuable insights for the development of marketing strategies.

show abstract

Similar and distinct neural mechanisms of visual and auditory emotion perception

Cited by 4 publications

References 29 publications

Weighted RSA: An Improved Framework on the Perception of Audio-visual Affective Speech in Left Insula and Superior Temporal Gyrus

Weighted RSA: An Improved Framework on the Perception of Audio-visual Affective Speech in Left Insula and Superior Temporal Gyrus

Weighted RSA: an improved framework on the perception of audio-visual affective speech in left insula and superior temporal gyrus

Effect of Interaction Effects Between Visual and Auditory Signs on Consumer Purchasing Behavior Based on AISAS Model

Contact Info

Product

Resources

About