Prediction Errors but Not Sharpened Signals Simulate Multivoxel fMRI Patterns during Speech Perception

Abstract

Successful perception depends on combining sensory input with prior knowledge. However, the underlying mechanism by which these two sources of information are combined is unknown. In speech perception, as in other domains, two functionally distinct coding schemes have been proposed for how expectations influence representation of sensory evidence. Traditional models suggest that expected features of the speech input are enhanced or sharpened via interactive activation (Sharpened Signals). Conversely, Predictive Coding suggests that expected features are suppressed so that unexpected features of the speech input (Prediction Errors) are processed further. The present work is aimed at distinguishing between these two accounts of how prior knowledge influences speech perception. By combining behavioural, univariate, and multivariate fMRI measures of how sensory detail and prior expectations influence speech perception with computational modelling, we provide evidence in favour of Prediction Error computations. Increased sensory detail and informative expectations have additive behavioural and univariate neural effects because they both improve the accuracy of word report and reduce the BOLD signal in lateral temporal lobe regions. However, sensory detail and informative expectations have interacting effects on speech representations shown by multivariate fMRI in the posterior superior temporal sulcus. When prior knowledge was absent, increased sensory detail enhanced the amount of speech information measured in superior temporal multivoxel patterns, but with informative expectations, increased sensory detail reduced the amount of measured information. Computational simulations of Sharpened Signals and Prediction Errors during speech perception could both explain these behavioural and univariate fMRI observations. However, the multivariate fMRI observations were uniquely simulated by a Prediction Error and not a Sharpened Signal model. The interaction between prior expectation and sensory detail provides evidence for a Predictive Coding account of speech perception. Our work establishes methods that can be used to distinguish representations of Prediction Error and Sharpened Signals in other perceptual domains.

Statistical learning and probabilistic prediction in music cognition: mechanisms of stylistic enculturation

“…Top‐down predictions are precision weighted such that more specific predictions (i.e., those more sharply focused on a single outcome) generate greater predictions errors. In the auditory modality, there is some evidence supporting hierarchical predictive coding for perception of nonmusical pitch sequences and speech, though not all aspects of the theory have been empirically substantiated . Vuust and colleagues have proposed a predictive coding theory of rhythmic incongruity …”

Flexible and adaptive processes in speech perception

“…Similar claims have been made in speech perception, based on findings that effects of contextual and acoustic manipulations permeate multiple levels of processing with effects of context also impacting early perceptual processes and effects of acoustic manipulations propagating to later stages of processing (Aydelott, Dick & Mills, 2006). There is evidence for modulation of activity in perceptual areas (Clos et al, 2012;Sohoglu & Davis, 2012;Wild & Davis, 2012;Blank & Davis, 2016). There is also evidence that interactions between multiple sources of information enhance speech intelligibility (Obleser et al, 2007(Obleser et al, , 2010Peelle et al, 2013;Gross et al, 2013).…”

“…A relative increase in activity for the biasing compared to neutral context (i. e., Sentence Context) emerged in two frontal clusters, whereas the effects of Target Type showed a relative increase in activity in inferior frontal gyrus (IFG), and a relative decrease in activity in STG. Enhanced activity for the biasing context and the ambiguous stimulus in the IFG could result from the effect of the semantic predictions that provided the source for top-down modulation and the resulting propagated error that denotes the differences between the expected and unexpected sensory input, generated in auditory cortex (Davis & Sohoglu, 2016;Blank & Davis, 2016). Enhanced activity for the ambiguous target in the IFG may be due to increased difficulty of category decisions (e. g., Binder et al, 2004;Myers et al, 2009).…”

“…Specifically, they used EEG to examine the effect of context (matching Brought to you by | New York University Authenticated Download Date | 4/23/17 2:56 AM vs. mismatching orthographic prime) on different levels of a degraded speech signal; they showed decreases in activity in perceptual areas (STG in their model) for both effects of predictive context and perceptual learning. The model accounts for both flexible and adaptive processes with a common computation that minimizes the prediction error (also see more recent paper by Blank & Davis, 2016).…”

“…However, even if a common mathematical computation can account for the two phenomena (e. g., Sohoglu & Davis, 2016;Blank & Davis, 2016), the underlying neural and biological mechanisms that support flexible vs. adaptive processes may differ. We proposed that the cerebellum contributes to adaptive changes in perception by using prediction errors, potentially at multiple levels of processing to guide learning (for alternative cerebellar role, see Lametti et al, 2016).…”