Predicting “When” in Discourse Engages the Human Dorsal Auditory Stream: An fMRI Study Using Naturalistic Stories

Abstract

The hierarchical organization of human cortical circuits integrates information across different timescales via temporal receptive windows, which increase in length from lower to higher levels of the cortical hierarchy (Hasson et al., 2015). A recent neurobiological model of higher-order language processing (Bornkessel-Schlesewsky et al., 2015) posits that temporal receptive windows in the dorsal auditory stream provide the basis for a hierarchically organized predictive coding architecture (Friston and Kiebel, 2009). In this stream, a nested set of internal models generates time-based (“when”) predictions for upcoming input at different linguistic levels (sounds, words, sentences, discourse). Here, we used naturalistic stories to test the hypothesis that multi-sentence, discourse-level predictions are processed in the dorsal auditory stream, yielding attenuated BOLD responses for highly predicted versus less strongly predicted language input. The results were as hypothesized: discourse-related cues, such as passive voice, which effect a higher predictability of remention for a character at a later point within a story, led to attenuated BOLD responses for auditory input of high versus low predictability within the dorsal auditory stream, specifically in the inferior parietal lobule, middle frontal gyrus, and dorsal parts of the inferior frontal gyrus, among other areas. Additionally, we found effects of content-related (“what”) predictions in ventral regions. These findings provide novel evidence that hierarchical predictive coding extends to discourse-level processing in natural language. Importantly, they ground language processing on a hierarchically organized predictive network, as a common underlying neurobiological basis shared with other brain functions.SIGNIFICANCE STATEMENT Language is the most powerful communicative medium available to humans. Nevertheless, we lack an understanding of the neurobiological basis of language processing in natural contexts: it is not clear how the human brain processes linguistic input within the rich contextual environments of our everyday language experience. This fMRI study provides the first demonstration that, in natural stories, predictions concerning the probability of remention of a protagonist at a later point are processed in the dorsal auditory stream. Results are congruent with a hierarchical predictive coding architecture assuming temporal receptive windows of increasing length from auditory to higher-order cortices. Accordingly, language processing in rich contextual settings can be explained via domain-general, neurobiological mechanisms of information processing in the human brain.

Contributions of local speech encoding and functional connectivity to audio-visual speech perception

“…Predictions modulate auditory evoked responses in an area specific manner, involve both the ventral and dorsal pathways (Kandylaki et al, 2016; Sohoglu and Chait, 2016), and affect both feedforward and feedback connections (Auksztulewicz and Friston, 2016; Chennu et al, 2016). While an informative visual context facilitates the correction of predictions about expected speech using incoming multisensory evidence, we can only speculate about a direct link between the reported effects and predictive processes.…”

Putting language back into ecological communication contexts

“…To fill this gap, recent language research has relied more on an ecological approach in which complex, naturalistic, real-life stimuli are employed to unveil the neural processes underlying speech comprehension. For example, stories have been frequently employed (Beeman et al, 2000;Ferstl, Rinck, & Cramon, 2005;Hartung, Hagoort, & Willems, 2017;Haupt, Schlesewsky, Roehm, Friederici, & Bornkessel-Schlesewsky, 2008;Kandylaki et al, 2016Kandylaki et al, , 2017 in both first and second language studies (Hsu, Jacobs, & Conrad, 2015). The degree of naturalness and complexity varies; for example, Haupt et al (2008) used very short stories and engaged participants in a comprehension task, while Kandylaki and colleagues employed longer texts in the absence of a specific task (Kandylaki et al, 2017).…”

“…Besides being closer to reallife story telling, stories also allow tackling mechanisms such as the processing of information spanning over longer time scalesthat short phrases would not allow identifying: As stories usually follow a coherent evolution, predictions on what is going to happen next in the narrative can be done based on the evolving context. This process has been explored in an fMRI study by Kandylaki and colleagues, who showed that this type of prediction engages the dorsal auditory stream, comparable with a hierarchical predictive coding architecture (Kandylaki et al, 2016; see also Willems, Frank, Nijhof, Hagoort, & Van Den Bosch, 2016). Such long-range effects could not be detected in classical experimental set-ups that would also not allow marking sustained effects that arise in an evolving story line which generates expectations (e.g.…”

“…Luckily however, the raising interest in interactive paradigms led researchers to find progressively more refined solutions to these issues anddependent on the purpose of the studythese aspects may be controlled in different Figure 1. Schematic representation of a classical language stimulus presented to a single participant (1: "the hungry cat chased the fast mouse", extracted from Friederici et al, 2000), and of two possible directions to increase its ecological validity: On the one hand, by shifting to a more complex stimulus embedding the target phrase (as in 2 - Haupt et al, 2008;Kandylaki et al, 2016); on the other, by shifting from the individual to the dyad (as in Kuhlen et al, 2017;Verga & Kotz, 2017). These two approaches may ultimately be combined into a complex setting reflecting an ecological interactive conversation (4).…”

No evidence for functional distinctions across fronto-temporal language regions in their temporal receptive windows

“…Nonetheless, the functional architecture of this network-i.e., the division of linguistic labor among its constituent regions-remains highly debated. On the one hand, some neuroimaging studies have suggested that different linguistic processes are localized to focal and distinct subsets of this network (e.g., Stowe et al, 1998;Vandenberghe et al, 2002;Bornkessel et al, 2005;Humphries et al, 2006;Caplan et al, 2008;Snijders et al, 2009;Meltzer et al, 2010;Pallier et al, 2011;Brennan et al, 2012;Goucha and Friederici, 2015;Zhang and Pylkkänen, 2015;Kandylaki et al, 2016;Frank and Willems, 2017;Wilson et al, 2018;Bhattasali et al, 2019). On the other hand, other studies have reported that such processes are widely distributed and spatially overlapping (e.g., Keller et al, 2001;Vigneau et al, 2006;Fedorenko et al, 2012b;Bautista and Wilson, 2016;Blank et al, 2016;Fedorenko et al, 2018;Siegelman et al, 2019).…”