High-order areas and auditory cortex both represent the high-level event structure of music

Abstract

Recent fMRI studies of event segmentation have found that default mode regions represent high-level event structure during movie watching. In these regions, neural patterns are relatively stable during events and shift at event boundaries. Music, like narratives, contains hierarchical event structure (e.g., sections are composed of phrases). Here, we tested the hypothesis that brain activity patterns in default mode regions reflect the high-level event structure of music. We used fMRI to record brain activity from 25 participants (male and female) as they listened to a continuous playlist of 16 musical excerpts, and additionally collected annotations for these excerpts by asking a separate group of participants to mark when meaningful changes occurred in each one. We then identified temporal boundaries between stable patterns of brain activity using a hidden Markov model and compared the location of the model boundaries to the location of the human annotations. We identified multiple brain regions with significant matches to the observer-identified boundaries, including auditory cortex, mPFC, parietal cortex, and angular gyrus. From these results, we conclude that both higher-order and sensory areas contain information relating to the high-level event structure of music. Moreover, the higher-order areas in this study overlap with areas found in previous studies of event perception in movies and audio narratives, including regions in the default mode network.Significance StatementListening to music requires the brain to track dynamics at multiple hierarchical timescales. In our study, we had fMRI participants listen to real-world music (classical and jazz pieces) and then used an unsupervised learning algorithm (a hidden Markov model) to model the high-level event structure of music within participants’ brain data. This approach revealed that default mode brain regions involved in representing the high-level event structure of narratives are also involved in representing the high-level event structure of music. These findings provide converging support for the hypothesis that these regions play a domain-general role in processing stimuli with long-timescale dependencies.

Information flow across the cortical timescales hierarchy during narrative construction

“…Our results are also consistent with reports on the spatiotemporal dynamics of brain responses to naturalistic stimuli. A hierarchically nested spatial activation pattern has been revealed using movie, spoken story, and music stimuli (22, 23, 47). Chien and colleagues (7) reported a gradual alignment of context-specific spatial activation patterns, which was rapidly flushed at event boundaries, similar to the temporal integration function we adopted here.…”

“…This study relied on eight openly available spoken story datasets. Seven datasets were used from the "Narratives" collection (OpenNeuro: https://openneuro.org/datasets/ds002245) (Nastase et al, 2021), including "Sherlock" and "Merlin" (18 participants, 11 females) (Zadbood et al, 2017), "The 21 st year" (25 participants, 14 females) (Chang et al, 2021), "Pie Man (PNI)", "I Knew You Were Black", "The Man Who Forgot Ray Bradbury", and "Running from the Bronx (PNI)" (48 participants, 34 females). One dataset was used from Princeton Dataspace: "Pie Man" (36 participants, 25 females) (https://dataspace.princeton.edu/jspui/handle/88435/dsp015d86p269k) (Simony et al, 2016).…”

“…Isolating stimulus-locked neural activity from intrinsic neural activity allows us for the first time to observe the propagation of linguistic information across the cortical hierarchy. We aggregated eight functional magnetic resonance imaging (fMRI) story-listening datasets (Nastase et al, 2021) to find the reliable, core sequence of information flow across a variety of diverse spoken story stimuli. Note that linguistic and narrative information unfolds over relatively long timescales: for example, single phonemes and words span hundreds of milliseconds, while phrases and paragraphs unfold over seconds and even minutes (Honey et al, 2012;Stephens et al, 2013).…”

Effects of familiar music exposure on deliberate retrieval of remote episodic and semantic memories in healthy aging adults

“…In line with these findings, some prior work has found that unfamiliar music elicits fewer autobiographical memories compared to environmental sounds or word cues, suggesting that unfamiliar music may not be a strong retrieval cue for many memories (Jakubowski & Eerola, 2021). Unfamiliar clips may shift focus away from retrieval and towards an "encoding mode" in which participants attend to sonic features, lyrics, or musical event structures (Janata, 2005;Janata et al, 2002;Williams et al, 2022). Further, participants in the current study may have focused their attention on trying to identify the unfamiliar music clips; this could have suppressed memory retrieval.…”

Music-selective cortex is sensitive to structure in both pitch and time

“…Music-selective neural populations in auditory cortex might thus be responsible for extracting temporally local features that are assembled elsewhere into more abstract representations of music, including key, meter, groove, event structure, etc. (Janata et al, 2002;Brett and Grahn, 2007;Lee et al, 2011;Fedorenko et al, 2012;Matthews et al, 2020;Williams et al, 2021). It is also plausible that responses might be further modulated by top-down inputs from brain regions like frontal cortex, perhaps reflecting the important role of expectation when it comes to music perception (Koelsch et al, 2018).…”