Decoding Temporal Structure in Music and Speech Relies on Shared Brain Resources but Elicits Different Fine-Scale Spatial Patterns

Abstract

Music and speech are complex sound streams with hierarchical rules of temporal organization that become elaborated over time. Here, we use functional magnetic resonance imaging to measure brain activity patterns in 20 right-handed nonmusicians as they listened to natural and temporally reordered musical and speech stimuli matched for familiarity, emotion, and valence. Heart rate variability and mean respiration rates were simultaneously measured and were found not to differ between musical and speech stimuli. Although the same manipulation of temporal structure elicited brain activation level differences of similar magnitude for both music and speech stimuli, multivariate classification analysis revealed distinct spatial patterns of brain responses in the 2 domains. Distributed neuronal populations that included the inferior frontal cortex, the posterior and anterior superior and middle temporal gyri, and the auditory brainstem classified temporal structure manipulations in music and speech with significant levels of accuracy. While agreeing with previous findings that music and speech processing share neural substrates, this work shows that temporal structure in the 2 domains is encoded differently, highlighting a fundamental dissimilarity in how the same neural resources are deployed.

Musical rhythm spectra from Bach to Joplin obey a 1/ f power law

“…One might argue that any sequence of musical rhythms displays 1/f structure and that our experiment lacks a control condition. To address this argument, we conducted additional analyses in which each stimulus served as its own control, disrupting the temporal structure of the rhythm at all frequencies (Materials and Methods) by shuffling the note onsets globally, across the piece, but keeping note durations intact (37,38). The spectrum of the resulting "shuffled" piece was flat and resembled white noise (β = 0.02) (Fig.…”

Neural overlap in processing music and speech

“…These findings highlight the existence of overlapping but distinct networks for music and speech within the same cortical areas. Similarly, Abrams et al [36] used multivariate pattern analysis for natural and scrambled music and speech excerpts and also found distinct brain patterns of responses to the two categories of sounds in several regions within the temporal lobe and the inferior frontal cortex. Therefore, the pattern of neural activation was distinct between music and speech, although there was overlap in the areas activated by the two domains.…”

“…It is important to point out that, even if the stimuli are matched for emotional content, attention, memory, subjective interest, arousal and familiarity [36], any observed category differences in activation strengths and/or patterns could be owing to acoustical differences. In order to avoid this confound, at least to some extent, one can use sung melodies and spoken lyrics from songs.…”

Processing structure in language and music: a case for shared reliance on cognitive control

“…Fedorenko & Kanwisher, 2009). In fact, most of the few recent studies that have included within-subjects comparisons of linguistic and musical manipulations have not found substantial overlap between neural regions implicated in the processing of language and music (but see Abrams et al, 2011). For example, Fedorenko and colleagues (Fedorenko, Behr, & Kanwisher, 2011;Fedorenko, McDermott, Norman-Haignere, & Kanwisher, 2012) used a contrast between intact sentences and lists of unconnected words (visually presented word-by-word) to define a series of language-sensitive brain regions of interest (ROIs) for each participant, and then investigated whether a musical manipulation significantly engaged those same regions.…”

“…In addition, manipulations of harmonic structure in fMRI paradigms show effects in brain areas typically associated with linguistic syntax including (most relevant to the following discussion) left inferior frontal regions, i.e., Broca's area (Janata, Tillmann, & Bharucha, 2002;Koelsch et al, 2002;Koelsch, Fritz, Schulze, Alsop, & Schlaug, 2005a;Minati et al, 2008;Oechslin, Van De Ville, Lazeyras, Hauert, & James, 2013;Tillmann, Janata, & Bharucha, 2003;Tillmann et al, 2006;Seger et al, 2013). These inferior frontal regions have also been implicated in the processing of rhythmic structure (Vuust, Roepstorff, Wallentin, Mouridsen, & Østergaard, 2006;Vuust, Wallentin, Mouridsen, Østergaard, & Roepstorff, 2011), and both frontal and temporal regions show equal sensitivity to temporal structure in music and speech (Abrams et al, 2011). Finally, there is a growing body of behavioral evidence linking the processing of musical and linguistic structure (e.g., Hoch, Poulin-Charronnat, & Tillmann, 2011;Fedorenko, Patel, Casasanto, Winawer, & Gibson, 2009;Slevc, Rosenberg, & Patel, 2009), as discussed below.…”