Role of Striatum in Updating Values of Chosen Actions

Abstract

The striatum is thought to play a crucial role in value-based decision making. Although a large body of evidence suggests its involvement in action selection as well as action evaluation, underlying neural processes for these functions of the striatum are largely unknown. To obtain insights on this matter, we simultaneously recorded neuronal activity in the dorsal and ventral striatum of rats performing a dynamic two-armed bandit task, and examined temporal profiles of neural signals related to animal's choice, its outcome, and action value. Whereas significant neural signals for action value were found in both structures before animal's choice of action, signals related to the upcoming choice were relatively weak and began to emerge only in the dorsal striatum ϳ200 ms before the behavioral manifestation of the animal's choice. In contrast, once the animal revealed its choice, signals related to choice and its value increased steeply and persisted until the outcome of animal's choice was revealed, so that some neurons in both structures concurrently conveyed signals related to animal's choice, its outcome, and the value of chosen action. Thus, all the components necessary for updating values of chosen actions were available in the striatum. These results suggest that the striatum not only represents values associated with potential choices before animal's choice of action, but might also update the value of chosen action once its outcome is revealed. In contrast, action selection might take place elsewhere or in the dorsal striatum only immediately before its behavioral manifestation.

Hippocampal Neural Correlates for Values of Experienced Events

“…There was a significant tendency for the neurons encoding choice outcome during the CS period to encode choice outcome during the first 1 s of the reward period as well in CA1 ( 2 test, p Ͻ 0.001), but not in the subiculum ( p ϭ 0.481). Consistent with the previous findings in the striatum (Roitman et al, 2005;Ito and Doya, 2009;Kim et al 2009), there was a significant tendency for choice outcome-encoding CA1 neurons to decrease their activity in rewarded trials. During the CS period, 7 and 14 CA1 neurons showed significantly higher and lower activity in rewarded trials, respectively, and this difference was not significant ( 2 test, p ϭ 0.127).…”

“…1 A) as described previously Kim et al, 2009;Sul et al, 2010). It was a free binary choice task with each choice associated with a different probability of reward that was constant within a block of trials, but changed across blocks with no explicit sensory cues.…”

Contributions of the nucleus accumbens and its subregions to different aspects of risk-based decision making

“…Rather, these findings suggest that under conditions where reward delivery occurs with varying probabilities, the NAc plays a key role in using information about recently rewarded actions to bias the direction of subsequent ones. This notion is in keeping with data from neurophysiological recording studies of ventral striatal neurons in rats performing a probabilistic reversal task, where firing in a proportion of these cells conveyed information about both current and previous choices while the animal approached the reward location (Kim, Sul, Huh, Lee, & Jung, 2009).…”

“…Nevertheless, these neurophysiological studies, in combination with the present data, make it reasonable to propose that phasic changes in NAc activity prior to a choice may serve to bias the direction of behavior (via descending projections that feed into motor systems) toward options associated with larger, probabilistic rewards that have greater long-term value. Moreover, changes in NAc activity preceding a choice may in turn be modulated by the outcomes of previous choices (Kim et al, 2009).…”

Reward Prediction Error Coding in Dorsal Striatal Neurons

“…Morris et al (2004) recorded the activity of TANs (tonically active neurons), or striatal cholinergic interneurons, in monkeys and concluded that TANs in monkeys do not code RPE, whereas Apicella et al (2009) have reported that they responded to RPE in a fashion inverse to that of dopamine neurons, that is, they show inhibitory activity for a positive RPE. Roesch et al (2009) recorded the activity of rat striatal neurons that were apparently medium spiny projection neurons and reported that only few of them appeared to code RPE signals, whereas Kim et al (2009) found both inverse and noninverse RPE coding in several medium spiny neurons and interneurons. The purpose of this study was to further examine whether RPE signals are coded in the striatum by using the behavioral paradigm of probabilistic pavlovian conditioning with which Fiorillo et al (2003) examined in detail the quantitative coding of RPE signals in monkey dopamine neurons.…”