Discovering Event Structure in Continuous Narrative Perception and Memory

Baldassano, Christopher; Chen, Janice; Zadbood, Asieh; Pillow, Jonathan W.; Hasson, Uri; Norman, Kenneth A.

doi:10.1016/j.neuron.2017.06.041

Cited by 647 publications

(940 citation statements)

References 78 publications

(116 reference statements)

Supporting

127

Mentioning

803

Contrasting

Unclassified

Order By: Relevance

“…The topographical hierarchy of processing timescales along the cortical surface (2) has been supported and replicated across methodologies, including single-unit analysis (4), electrocorticography analysis (7), fMRI analysis (5,6,25), magnetoencephalography analysis (1), computational models (3), and resting state functional connectivity (30,31). This hierarchy, as well as the divergence of neural responses observed here, is additionally consistent with previously proposed linguistic hierarchies: low-level regions (A1+) represent phonemes (32,33), syllables (34), and pseudowords (35), while medium-level regions (areas along A1+ to STS) represent sentences (36,37).…”

Section: Discussionmentioning

confidence: 99%

Amplification of local changes along the timescale processing hierarchy

Yeshurun

Nguyen

Hasson

2017

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

Small changes in word choice can lead to dramatically different interpretations of narratives. How does the brain accumulate and integrate such local changes to construct unique neural representations for different stories? In this study, we created two distinct narratives by changing only a few words in each sentence (e.g., "he" to "she" or "sobbing" to "laughing") while preserving the grammatical structure across stories. We then measured changes in neural responses between the two stories. We found that differences in neural responses between the two stories gradually increased along the hierarchy of processing timescales. For areas with short integration windows, such as early auditory cortex, the differences in neural responses between the two stories were relatively small. In contrast, in areas with the longest integration windows at the top of the hierarchy, such as the precuneus, temporal parietal junction, and medial frontal cortices, there were large differences in neural responses between stories. Furthermore, this gradual increase in neural differences between the stories was highly correlated with an area's ability to integrate information over time. Amplification of neural differences did not occur when changes in words did not alter the interpretation of the story (e.g., sobbing to "crying"). Our results demonstrate how subtle differences in words are gradually accumulated and amplified along the cortical hierarchy as the brain constructs a narrative over time.timescale | amplification | fMRI | narrative | hierarchy S tories unfold over many minutes and are organized into temporarily nested structures: Paragraphs are made of sentences, which are made of words, which are made of phonemes. Understanding a story therefore requires processing the story at multiple timescales, starting with the integration of phonemes to words within a relatively short temporal window, to the integration of words to sentences across a longer temporal window of a few seconds, and up to the integration of sentences and paragraphs into a coherent narrative over many minutes. It was recently suggested that the temporally nested structure of language is processed hierarchically along the cortical surface (1-4). A consequence of the hierarchical structure of language is that small changes in word choice can give rise to large differences in sentence and overall narrative interpretation. Here, we propose that local momentary changes in linguistic input in the context of a narrative (e.g., "he" to "she" or "sobbing" to "laughing") are accumulated and amplified during the processing of linguistic content across the cortex. Specifically, we hypothesize that as areas in the brain increase in their ability to integrate information over time (e.g., processing "word" level content vs. "sentence" or "narrative" level content), the neural response to small word changes will become increasingly divergent.Previously, we defined a temporal receptive window (TRW) as the length of time during which prior information from an ongoing stimulu...

show abstract

Section: Discussionmentioning

confidence: 99%

Amplification of local changes along the timescale processing hierarchy

Yeshurun

Nguyen

Hasson

2017

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The largest axis of variation separates perceptual and physical categories in sensorimotor areas from more abstract concepts in transmodal regions [46]. (D) The length of events that are represented in a given area, here extracted from movie-watching data, varies from short events in sensory areas to long events in transmodal regions (only patterns with high between-subject consistency are shown, for example, somatosensory regions did not respond consistently to auditory-visual input) [53].…”

Section: Glossarymentioning

confidence: 99%

“…A recent line of research introduced the concept of temporal receptive windows -in analogy to spatial receptive fields -reflecting the time window in which previously presented information can affect the processing of a newly arriving stimulus [48][49][50][51][52][53]. The length of temporal receptive fields was found to vary hierarchically from primary sensory areas, tracking fast changes of a scene on the order of milliseconds to seconds, to transmodal association areas, which encode slowly changing states of the world, complex concepts and situations, and integrate information across seconds, minutes, or longer ( Figure 1D).…”

Section: A Temporal Hierarchy Links Structural Gradients and Functionmentioning

confidence: 99%

Large-Scale Gradients in Human Cortical Organization

Huntenburg

Bazin

Margulies

2018

Trends in Cognitive Sciences

723

733

View full text Add to dashboard Cite

Recent advances in mapping cortical areas in the human brain provide a basis for investigating the significance of their spatial arrangement. Here we describe a dominant gradient in cortical features that spans between sensorimotor and transmodal areas. We propose that this gradient constitutes a core organizing axis of the human cerebral cortex, and describe an intrinsic coordinate system on its basis. Studying the cortex with respect to these intrinsic dimensions can inform our understanding of how the spectrum of cortical function emerges from structural constraints. The Significance of Cortical LocationFor more than a century, neuroscientists have studied the cerebral cortex by delineating individual cortical areas (see Glossary) and mapping their function [1]. This agenda has substantially advanced in recent years, as automated parcellation methods improve and data sets of unprecedented size and quality become available [2][3][4]. Nevertheless, our understanding of how the complex structure of the cerebral cortex emerges and gives rise to its elaborate functions remains fragmentary. To complement the description of individual cortical areas, we propose an inquiry into the significance of their spatial arrangement, asking the basic question: Why are cortical areas located where they are?Early formulations of this question date to theories from classical neuroanatomy [1,[5][6][7]. They state that the spatial layout of cortical areas is not arbitrary, but a consequence of developmental mechanisms, shaped through evolutionary selection. The location of an area among its neighbors thus provides insight into its microstructural characteristics [6], its connections to other parts of the brain [7], and eventually its position in global processing hierarchies [8]. Consider, for example, the well-researched visual system of the macaque monkey [9,10]. Along the visual hierarchy, low-level visual features are increasingly abstracted and integrated with information from other systems. Traditionally, areas are ordered based on their degree of microstructural differentiation, and the classification of their connections as feedforward or feedback [11]. The framework we advocate emphasizes that an area's position in the visual processing hierarchy -and thus many of its microstructural and connectional features -is strongly related to its distance from the primary visual area [12,13].More generally, we propose that the spatial arrangement of areas along a global gradient between sensorimotor and transmodal regions is a key feature of human cortical organization. A gradient is an axis of variance in cortical features, along which areas fall in a spatially continuous order. Areas that resemble each other with respect to the feature of interest occupy similar positions along the gradient. As we will point out, there is a strong relationship between the similarity of two areas -that is, their relative position along the gradient -and their relative position along the cortical surface. This important role of cortical location pro...

show abstract

“…The inflation of pre-boundary temporal information and increased subjective recollection may be produced by a post-boundary peak in hippocampal activity, which enables binding of preboundary information into a cohesive event (Baldassano et al, 2017;Ben-Yakov & Dudai, 2011;Ben-Yakov, Eshel, & Dudai, 2013). This retroactive signal might therefore result in an enrichment of pre-boundary, relative to post-boundary information.…”

Section: Discussionmentioning

confidence: 99%

“…A computational model of this cross-boundary memory disruption suggests that the rate of temporal context drift is increased immediately following boundaries (Horner et al, 2016), effectively adding noise to information essential for temporal order judgements. However, when information on either side of a boundary represents a cohesive unit, boundaries actually improve memory by providing structure (Pettijohn, Thompson, Tamplin, Krawietz, & Radvansky, 2016), which might be supported by a peak in hippocampal activity between events (Baldassano et al, 2017;Ben-Yakov & Dudai, 2011). Boundaries therefore differentially affect different memory aspects.…”

Section: Introductionmentioning

confidence: 99%

Does a turn in the road mark a turn of events? Turns along travelled routes provide contextual boundaries during navigation

Brunec¹,

Ozubko²,

Ander³

et al. 2018

Preprint

View full text Add to dashboard Cite

Physical or conceptual boundaries segment continuous experience into component events. Boundaries accentuate differences between events that occur on either side of them, and reduce differences among events within them. Here we show that turns during navigation have the same effect on memory for routes as they do for episodes. Across three experiments, turns selectively enhanced participants' recollection of locations immediately preceding them. In addition, in Experiments 1 and 2, the presence of an intervening turn enhanced participants' ability to discriminate between event durations across boundaries, but impaired their ability to discriminate between their ordinal positions. In Experiment 3, the reported increase in recollection of pre-turn locations was also reflected in subjective dilation of the time spent at preturn, relative to post-turn, locations. Together, these results highlight the fundamental role of turns in the segmentation of spatial and temporal memory and indicate a potential mechanism for the segmentation of experience generally.

show abstract

Discovering Event Structure in Continuous Narrative Perception and Memory

Cited by 647 publications

References 78 publications

Amplification of local changes along the timescale processing hierarchy

Amplification of local changes along the timescale processing hierarchy

Large-Scale Gradients in Human Cortical Organization

Does a turn in the road mark a turn of events? Turns along travelled routes provide contextual boundaries during navigation

Contact Info

Product

Resources

About