“What” and “when” predictions jointly modulate speech processing

Abstract

Adaptive behavior rests on forming predictions based on previous statistical regularities encountered in the environment. Such regularities pertain not only to the contents of the stimuli ("what") but also their timing ("when"), and both interactively modulate sensory processing. In speech streams, predictions can be formed at multiple hierarchical levels, both in terms of contents (e.g. single syllables vs. words) and timing (e.g., faster vs. slower time scales). Whether and how these hierarchies map onto each other in terms of integrating "what" and "when" predictions remains unknown. Under one hypothesis neural hierarchies may link "what" and "when" predictions within sensory processing areas: with lower cortical regions mediating interactions for smaller units e.g., syllables, and higher cortical areas mediating interactions for larger units e.g., words. Alternatively, interactions between "what" and "when" predictions might rest on a generic, sensory-independent mechanism, mediated by common attention-related (e.g., frontoparietal) networks. To address those questions, we manipulated "what" and "when" predictions at two levels - single syllables and disyllabic pseudowords - while recording neural activity using magnetoencephalography (MEG) in healthy volunteers (N=22). We studied how syllable and/or word deviants are modulated by "when" predictability, both analyzing event-related fields and using source reconstruction and dynamic causal modeling to explain the observed effects in terms of the underlying effective connectivity. "When" predictions modulated "what" mismatch responses in a specific way with regards to speech hierarchy, such that mismatch responses to deviant words (vs. syllables) were amplified by temporal predictions at a slower (vs. faster) time scale. However, these modulations were source-localized to a shared network of cortical regions, including frontal and parietal sources. Effective connectivity analysis showed that, while mismatch responses to violations of "what" predictions modulated connectivity between regions, the integration of "what" and "when" predictions selectively modulated connectivity within regions, consistent with gain effects. These results suggest that the brain integrates "what" and "when" predictions that are congruent with respect to their hierarchical level, but this integration is mediated by a shared and distributed cortical network. This contrasts with recent studies indicating separable networks for different levels of hierarchical speech processing.