Different Neural Correlates of Reward Expectation and Reward Expectation Error in the Putamen and Caudate Nucleus During Stimulus-Action-Reward Association Learning

Abstract

Haruno, Masahiko and Mitsuo Kawato. Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. J Neurophysiol 95: 948 -959, 2006. First published October 5, 2005 doi:10.1152/jn.00382.2005. To select appropriate behaviors leading to rewards, the brain needs to learn associations among sensory stimuli, selected behaviors, and rewards. Recent imaging and neural-recording studies have revealed that the dorsal striatum plays an important role in learning such stimulus-action-reward associations. However, the putamen and caudate nucleus are embedded in distinct cortico-striatal loop circuits, predominantly connected to motor-related cerebral cortical areas and frontal association areas, respectively. This difference in their cortical connections suggests that the putamen and caudate nucleus are engaged in different functional aspects of stimulus-action-reward association learning. To determine whether this is the case, we conducted an event-related and computational model-based functional MRI (fMRI) study with a stochastic decision-making task in which a stimulus-action-reward association must be learned. A simple reinforcement learning model not only reproduced the subject's action selections reasonably well but also allowed us to quantitatively estimate each subject's temporal profiles of stimulus-action-reward association and reward-prediction error during learning trials. These two internal representations were used in the fMRI correlation analysis. The results revealed that neural correlates of the stimulus-action-reward association reside in the putamen, whereas a correlation with reward-prediction error was found largely in the caudate nucleus and ventral striatum. These nonuniform spatiotemporal distributions of neural correlates within the dorsal striatum were maintained consistently at various levels of task difficulty, suggesting a functional difference in the dorsal striatum between the putamen and caudate nucleus during stimulus-action-reward association learning.

Altered emotional and BOLD responses to negative, positive and ambiguous performance feedback in OCD

“…Next to its role in the selection, execution and evaluation of motor programs, the putamen features prominently in cognitive and motivational theories. It has been implied in the anticipation and evaluation of reward-related feedback, particularly in elicitation of a neural response to reward prediction errors (Haruno and Kawato, 2006). The putamen has been suggested to be important for integration of stimulus-action-reward associations (Haruno and Kawato, 2006) and might mediate action selection by computing and evaluating their values in terms of sensory contexts and rewards.…”

“…It has been implied in the anticipation and evaluation of reward-related feedback, particularly in elicitation of a neural response to reward prediction errors (Haruno and Kawato, 2006). The putamen has been suggested to be important for integration of stimulus-action-reward associations (Haruno and Kawato, 2006) and might mediate action selection by computing and evaluating their values in terms of sensory contexts and rewards. Therefore, the lack of a differential response of the putamen in OCD patients to different kinds of feedback could indicate insensitivity to action outcomes in the context of future action selection.…”

Predictability of what or where reduces brain activity, but a bottleneck occurs when both are predictable

“…Based on prior work on coding of regularity we selected the bilateral thalamus, caudate, putamen, pallidum, hippocampus and nucleus accumbens as regions of interest. These were selected based on prior work linking them to the coding of regularities (Haruno & Kawato, 2006;Reddy et al, 2015;Turk-Browne, Scholl, Johnson, & Chun, 2010). Subcortical ROIs were transformed from common MNI space to original space using the inverse of the transformation we derived to convert them from original space to MNI space.…”

Visual cortex signals a mismatch between regularity of auditory and visual streams

“…The putamen and caudate are thought to play a role in the construction of prediction and calculation of prediction error (e.g., Haruno & Kawato, 2006), though their role in prediction of auditory inputs is less clear Turk-Browne et al, 2010). The hippocampus has been suggested to be an important mediator of statistical learning, given its involvement in associative learning (e.g., Turk-Browne et al, 2010), and demonstrations of its sensitivity to input uncertainty (e.g., Harrison, Duggins, & Friston, 2006;Strange et al, 2005).…”