The Music of Your Emotions: Neural Substrates Involved in Detection of Emotional Correspondence between Auditory and Visual Music Actions

Abstract

In humans, emotions from music serve important communicative roles. Despite a growing interest in the neural basis of music perception, action and emotion, the majority of previous studies in this area have focused on the auditory aspects of music performances. Here we investigate how the brain processes the emotions elicited by audiovisual music performances. We used event-related functional magnetic resonance imaging, and in Experiment 1 we defined the areas responding to audiovisual (musician's movements with music), visual (musician's movements only), and auditory emotional (music only) displays. Subsequently a region of interest analysis was performed to examine if any of the areas detected in Experiment 1 showed greater activation for emotionally mismatching performances (combining the musician's movements with mismatching emotional sound) than for emotionally matching music performances (combining the musician's movements with matching emotional sound) as presented in Experiment 2 to the same participants. The insula and the left thalamus were found to respond consistently to visual, auditory and audiovisual emotional information and to have increased activation for emotionally mismatching displays in comparison with emotionally matching displays. In contrast, the right thalamus was found to respond to audiovisual emotional displays and to have similar activation for emotionally matching and mismatching displays. These results suggest that the insula and left thalamus have an active role in detecting emotional correspondence between auditory and visual information during music performances, whereas the right thalamus has a different role.

Dance as a subject for empirical aesthetics.

“…This suggests that its role during the aesthetic experience of dance may be related with the observer's motor simulation response. Activity in the inferior parietal gyrus and the occipitotemporal cortex (EBA) has also been associated with the processing of emotional body movements, such as emotional point light displays of emotions, emotional gestures, or the emotional movements of a musician (without music) Peelen, Atkinson, Andersson, & Vuilleumier, 2007;Petrini, et al, 2011). Surprisingly, however, activity in classical emotion-processing regions, such as the insula, the orbitofrontal cortex, and the anterior cingulate cortex (Adolphs, 2002;Keysers, Kaas, & Gazzola, 2010;Petrini, et al, 2011), has so far been reported only for tasks of dance observation (Cross, Kirsch, Ticini, & Schuetz-Bosbach, 2011;Jang & Pollick, 2011).…”

“…Activity in the inferior parietal gyrus and the occipitotemporal cortex (EBA) has also been associated with the processing of emotional body movements, such as emotional point light displays of emotions, emotional gestures, or the emotional movements of a musician (without music) Peelen, Atkinson, Andersson, & Vuilleumier, 2007;Petrini, et al, 2011). Surprisingly, however, activity in classical emotion-processing regions, such as the insula, the orbitofrontal cortex, and the anterior cingulate cortex (Adolphs, 2002;Keysers, Kaas, & Gazzola, 2010;Petrini, et al, 2011), has so far been reported only for tasks of dance observation (Cross, Kirsch, Ticini, & Schuetz-Bosbach, 2011;Jang & Pollick, 2011). Activity in such regions has not been found to be especially relevant in tasks of aesthetic appreciation of dance movements, which may be puzzling considering the rather large amount of empirical and theoretical work linking emotional and affective processes with art appreciation (Leder, Belke, Oeberst, & Augustin, 2004;Aharon, Etcoff, Ariely, Chabris, O'Connor, Breiter, 2001;Blood & Zatorre, 2001;Brown, Martinez, & Parsons, 2004;Vartanian & Goel, 2004).…”

How Bodies and Voices Interact in Early Emotion Perception

“…Not surprisingly, early interactions of body and vocal expressions are reliably observed [6], [7], and an interaction has also been reported between body motion and music produced by the very body motion [8], [9]. However, do these interactions truly reflect integration [10]?…”

Dynamic music and immersion in the action-adventure an empirical investigation

“…Characterized by superior temporal resolutions of expressive qualities, sonic dimensions, like music, are highly suitable to take over other senses during the initial direction of selective attention towards congruent perceptions [43] [26]. Thus, situation-specific associative and expectation driven evaluative functions of music influence the scenic interpretation in accordance to audio-visual correspondence as well as visual perception [6] [10] [33]. When expressive qualities of music stimuli become salient, multisensory expectations on emotional congruency towards the environment, termed hypotheses of perception, are formed.…”