Dynamic Emotional and Neural Responses to Music Depend on Performance Expression and Listener Experience

Abstract

Apart from its natural relevance to cognition, music provides a window into the intimate relationships between production, perception, experience, and emotion. Here, emotional responses and neural activity were observed as they evolved together with stimulus parameters over several minutes. Participants listened to a skilled music performance that included the natural fluctuations in timing and sound intensity that musicians use to evoke emotional responses. A mechanical performance of the same piece served as a control. Before and after fMRI scanning, participants reported real-time emotional responses on a 2-dimensional rating scale (arousal and valence) as they listened to each performance. During fMRI scanning, participants listened without reporting emotional responses. Limbic and paralimbic brain areas responded to the expressive dynamics of human music performance, and both emotion and reward related activations during music listening were dependent upon musical training. Moreover, dynamic changes in timing predicted ratings of emotional arousal, as well as real-time changes in neural activity. BOLD signal changes correlated with expressive timing fluctuations in cortical and subcortical motor areas consistent with pulse perception, and in a network consistent with the human mirror neuron system. These findings show that expressive music performance evokes emotion and reward related neural activations, and that music's affective impact on the brains of listeners is altered by musical training. Our observations are consistent with the idea that music performance evokes an emotional response through a form of empathy that is based, at least in part, on the perception of movement and on violations of pulse-based temporal expectancies.

Hidden sources of joy, fear, and sadness: Explicit versus implicit neural processing of musical emotions

“…Interestingly, activity in the inferior parietal lobule is consistent with explicit processing of emotional vocal and visual stimuli (Wildgruber et al, 2005;Scheuerecker et al, 2007) and with the implicit processing of prosody (Bach et al, 2008;Wildgruber et al, 2005). The right inferior parietal lobule has also been associated with musical emotion processing, particularly with respect to exciting pleasant versus relaxing pleasant music (Flores-Gutierrez et al, 2007) and with music performer expressivity (Chapin et al, 2010). A patient with an infarct in the right inferior parietal lobule reported musical anhedonia, or, in other words, the inability to have emotional experiences in response to music (Satoh et al, 2011).…”

Activity Analysis and Coordination in Continuous Responses to Music

“…Nielsen (1987) used only the average tension-rating time series to describe or connect ratings with the presented music. Others have clarified or reduced average series with down-sampling (Chapin, Jantzen, Kelso, Steinberg & Large, 2010) or by taking the first-order difference (Schubert, 2004). Time series analysis (Bailes & Dean, 2012), from the tradition of econometrics, and functional data analysis (Levitin, Nuzzo, Vines, & Ramsay, 2007), out of mathematical biology, have demonstrated the insights such tools support.…”

Modeling Perceptions of Valence in Diverse Music

“…As we have noted already, one subcomponent that it seems to represent is agency, in the specific sense of the apparent presence or absence of a particular human activity (such as vocal sounds, or a concerto soloist), and possibly the distinct narrative agency of musical development (such as perhaps exposition, development, or tonal shift). Another subcomponent is rhythmic flexibility, which has been proposed to be a predictor (Chapin, Jantzen, Kelso, Steinberg, & Large, 2010), but not previously analyzed in our studies. In Chapin et al (2010), removing expressive timing and dynamic variations together resulted in diminished affective responses, so their relative effects were not discriminated.…”

“…Another subcomponent is rhythmic flexibility, which has been proposed to be a predictor (Chapin, Jantzen, Kelso, Steinberg, & Large, 2010), but not previously analyzed in our studies. In Chapin et al (2010), removing expressive timing and dynamic variations together resulted in diminished affective responses, so their relative effects were not discriminated. Thus in this section we assess whether the agency of the soloist in our Mozart piano concerto extract can be quantitated, so as to provide a predictor which enhances our modeling of perceived arousal and valence.…”