Surprised at All the Entropy: Hippocampal, Caudate and Midbrain Contributions to Learning from Prediction Errors

Abstract

Influential concepts in neuroscientific research cast the brain a predictive machine that revises its predictions when they are violated by sensory input. This relates to the predictive coding account of perception, but also to learning. Learning from prediction errors has been suggested for take place in the hippocampal memory system as well as in the basal ganglia. The present fMRI study used an action-observation paradigm to investigate the contributions of the hippocampus, caudate nucleus and midbrain dopaminergic system to different types of learning: learning in the absence of prediction errors, learning from prediction errors, and responding to the accumulation of prediction errors in unpredictable stimulus configurations. We conducted analyses of the regions of interests' BOLD response towards these different types of learning, implementing a bootstrapping procedure to correct for false positives. We found both, caudate nucleus and the hippocampus to be activated by perceptual prediction errors. The hippocampal responses seemed to relate to the associative mismatch between a stored representation and current sensory input. Moreover, its response was significantly influenced by the average information, or Shannon entropy of the stimulus material. In accordance with earlier results, the habenula was activated by perceptual prediction errors. Lastly, we found that the substantia nigra was activated by the novelty of sensory input. In sum, we established that the midbrain dopaminergic system, the hippocampus, and the caudate nucleus were to different degrees significantly involved in the three different types of learning: acquisition of new information, learning from prediction errors and responding to unpredictable stimulus developments. We relate learning from perceptual prediction errors to the concept of predictive coding and related information theoretic accounts.

The role of prediction and outcomes in adaptive cognitive control

“…It is not yet conclusively established that striatal prediction errors are observed when subjects fail to reach anticipated subgoals (end states of options) that never entail reward delivery, and do not change the overall estimate of reward likelihood. However, a few studies investigating prediction errors in perception have yielded evidence that the striatum codes for the unexpectedness of events per se (den Ouden et al, 2009;Grahn et al, 2008;Grahn and Rowe, 2013;Schiffer and Schubotz, 2011;Schiffer et al, 2012;Seger et al, 2013) and is not limited to reward -related prediction error coding. Although unexpected events in these studies were not predictive of forthcoming reward, or positive feedback, they were sometimes task-relevant (e.g., Schiffer and Schubotz, 2011), even if only to the degree that they informed participants that they should pay attention to deviations in a stimulus to increase their ability to answer (unrewarded) questions correctly (Schiffer et al, 2012).…”

“…These findings warrant further research, not least because the role that specific frontal areas play in this rostro-caudal axis of the fronto-striatal loop remain a matter of debate: Very similar functions have been ascribed to different cortical areas (e.g., in Wilson et al, 2014) and, conversely, dissimilar functions have been proposed for nearly identical areas of cortex (compare for example Derrfuss et al, 2005). Further, while some studies point towards involvement of the basal ganglia in sensory predictions and sensory prediction-error coding (den Ouden et al, 2009;Grahn and Rowe, 2013;Schiffer and Schubotz, 2011;Schiffer et al, 2012), it is yet to be tested empirically whether neural networks involved in HRL support outcome prediction and action selection in non-reward contexts as proposed. Each of these questions of functional neuroanatomy needs to be followed up in future research.…”

Causal Mathematical Logic as a guiding framework for the prediction of “Intelligence Signals” in brain simulations

“…Also the thermodynamic casual approach to the neuroscience we highlight has existing ground, in the work of (Papo, 2013) and (Friston, Daunizeau and Kilner, 2010;Friston, Harrison, and Penny, 2003). There have also been recent questions raised on why fMRI of the brains prediction mechanisms in the limbic system appears to be driving us towards high entropy information (Davis, Love and Preston, 2012) with some degree of coding of entropic information occurring in the process (Schiffer, 2012). More recently (Carhart-harris et al, 2014) use causality based fMRI/MEG tools to argue that entropy is a primary (lower) component of consciousness.…”

Frontostriatal Contribution to the Interplay of Flexibility and Stability in Serial Prediction

“…Temporary extended increases in prediction errors indicate volatile environments and lead to a fast adjustment of internal predictions and behavior ( Jiang, Beck, Heller, & Egner, 2015;Chumbley et al, 2014;Schiffer, Ahlheim, Wurm, & Schubotz, 2012;Friston, Daunizeau, & Kiebel, 2009;Behrens, Woolrich, Walton, & Rushworth, 2007). Recent studies provide evidence that longer timescale tonic DA modulates the extent to which prior action outcome biases phasic DA release and hence future action selection ( Yu, FitzGerald, & Friston, 2013;Humphries, Khamassi, & Gurney, 2012;Beeler, Daw, Frazier, & Zhuang, 2010).…”

“…In contrast, stabilization of prediction was hypothesized to be accompanied by premotor and lateral pFC activation (Cohen, Braver, & Brown, 2002;Miller & Cohen, 2001), whereby either dorsal or ventral activation would reflect the way a model content is stored in working memory, that is, spatially or verbally, respectively (Rottschy et al, 2012). With regard to the role of the striatum in immediate response selection, we hypothesized caudate activity in response to both switches and drifts (Schiffer et al, 2012). Crucially, we expected the degree of activation increase to be correlated with the ability to discriminate between both events, showing that the striatum is related to selecting correct responses toward different types of prediction errors (Chatham & Badre, 2015;Badre, 2012).…”