Representation of Action-Specific Reward Values in the Striatum

Abstract

The estimation of the reward an action will yield is critical in decision-making. To elucidate the role of the basal ganglia in this process, we recorded striatal neurons of monkeys who chose between left and right handle turns, based on the estimated reward probabilities of the actions. During a delay period before the choices, the activity of more than one-third of striatal projection neurons was selective to the values of one of the two actions. Fewer neurons were tuned to relative values or action choice. These results suggest representation of action values in the striatum, which can guide action selection in the basal ganglia circuit.

Opponent actor learning (OpAL): Modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive.

“…The effective apparent "cost" of effort is bidirectionally modulated by manipulation of indirect D2 MSN activity: Pharmacological manipulations that enhance such activity result in more avoidance of effortful actions, whereas inhibition of this pathway has the opposite effect, decreasing the effective cost (Farrar et al, 2010Mingote et al, 2008;Nunes et al, 2010). Neural models suggest that these effects are mediated by differential coding of positive and negative consequences of actions in distinct MSN populations, as observed in electrophysiological studies (Samejima et al, 2005).…”

“…The effective apparent "cost" of effort is bidirectionally modulated by manipulation of indirect D2 MSN activity: Pharmacological manipulations that enhance such activity result in more avoidance of effortful actions, whereas inhibition of this pathway has the opposite effect, decreasing the effective cost (Farrar et al, 2010Mingote et al, 2008;Nunes et al, 2010). Neural models suggest that these effects are mediated by differential coding of positive and negative consequences of actions in distinct MSN populations, as observed in electrophysiological studies (Samejima et al, 2005).Recent optogenetic work (Tai, Lee, Benavidez, Bonci, & Wilbrecht, 2012) has also confirmed more precisely that specific action values can be inflated or diminished by stimulating D1 or D2 MSNs during the choice period (as opposed to during the outcome in the learning study described previously). Stimulating This document is copyrighted by the American Psychological Association or one of its allied publishers.…”

Serotonin and Dopamine: Unifying Affective, Activational, and Decision Functions

“…The predominant view in computational and systems neuroscience holds that DA serves to promote RL, that is, trial-and-error instrumental learning, to choose rewarding actions (Houk et al, 1995;Montague et al, 1996;Schultz et al, 1997;Samejima et al, 2005;Morris et al, 2006). This idea is derived from electrophysiological recordings from neurons in the midbrain dopaminergic nuclei of primates performing simple tasks for reward (Ljungberg et al, 1991;Hollerman and Schultz, 1998;Waelti et al, 2001), together with the insight that the phasic firing of these neurons quantitatively resembles a 'reward prediction error' signal used in computational algorithms for RL to improve action choice so as to obtain more rewards (Sutton and Barto, 1990;Montague et al, 1996;Sutton and Barto, 1998;Montague et al, 2004;Bayer and Glimcher, 2005;Frank, 2005).…”

Discovery of a novel nicotinic receptor antagonist for the treatment of nicotine addiction: 1-(3-Picolinium)-12-triethylammonium-dodecane dibromide (TMPD)

“…In our search for novel nicotinic receptor antagonists, the nicotine-evoked DA release assay is used as an initial screen to identify lead compounds, since the ability of nicotine to release DA is believed to be associated with the rewarding properties of nicotine, and striatum is important for identifying and anticipating rewards and organizing goal directed behavior [78][79][80][81]. TMPD was identified as a hit in this screen because it produced greater than 40% inhibition of nicotine-evoked DA release at a single probe concentration of 100 nM.…”