Basal Ganglia Orient Eyes to Reward

Hikosaka, Okihide; Nakamura, Kae; Nakahara, Hiroyuki

doi:10.1152/jn.00458.2005

Cited by 345 publications

(361 citation statements)

References 191 publications

Supporting

Mentioning

335

Contrasting

Unclassified

Order By: Relevance

“…Whereas our results do not support a direct action selection role of the striatum, they are consistent with the view that the striatum contributes to action selection by biasing animal's choice toward an action that is associated with a higher value (Samejima et al, 2005;Hikosaka et al, 2006). We found significant levels of action value signals before animal's choice of action, which is consistent with previous reports on the monkey DS (Samejima et al, 2005;Lau and Glimcher, 2008; also see Pasquereau et al, 2007).…”

Section: Role Of Striatum In Action Selectionsupporting

confidence: 91%

“…Our results also show that choice, reward and chosen value signals are all stronger in the DS, suggesting that the DS might play a more prominent role in updating value than the VS. Combined with recent studies (for review, see Hikosaka et al, 2006;Schultz, 2006; see also Kimchi and Laubach, 2009), our results clearly argue against the view that the DS and VS are exclusively involved in motor/cognitive functions and reward processing, respectively. It remains to be determined what specific roles the DS and VS play, and how they work together in evaluating past choices and determining the desirability of future actions.…”

Section: Functions Of Dorsal Versus Ventral Striatumsupporting

confidence: 58%

“…According to a popular view (Alexander and Crutcher, 1990a;DeLong, 2000), the BG and cortex form parallel loops serving different functions, and the VS, as part of the motivational loop, is primarily involved in reward processing and motivational control of behavior (Mogenson et al, 1980;Salamone and Correa, 2002;O'Doherty, 2004). However, recent studies have found strong reward-and value-related neural activity in the DS (for review, see Daw and Doya, 2006;Hikosaka et al, 2006;Schultz, 2006), indicating that the DS is also heavily involved in reward and value processing. Our results also show that choice, reward and chosen value signals are all stronger in the DS, suggesting that the DS might play a more prominent role in updating value than the VS.…”

Section: Functions Of Dorsal Versus Ventral Striatummentioning

confidence: 99%

“…These features of the BG led to the proposal that its primary role is selecting a specific action out of multiple candidates that are provided by the cortex and relaying this information to the downstream motor structures (Alexander and Crutcher, 1990a;Hikosaka, 1994;Mink, 1996;Redgrave et al, 1999). Recently, a revised theory has been proposed that the BG contribute to action selection by biasing its activity toward an action with the most desirable outcome (Hollerman and Schultz, 1998;Kawagoe et al, 1998;Samejima et al, 2005;Hikosaka et al, 2006).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Role of Striatum in Updating Values of Chosen Actions

Kim

Sul²,

Huh³

et al. 2009

J. Neurosci.

189

304

View full text Add to dashboard Cite

The striatum is thought to play a crucial role in value-based decision making. Although a large body of evidence suggests its involvement in action selection as well as action evaluation, underlying neural processes for these functions of the striatum are largely unknown. To obtain insights on this matter, we simultaneously recorded neuronal activity in the dorsal and ventral striatum of rats performing a dynamic two-armed bandit task, and examined temporal profiles of neural signals related to animal's choice, its outcome, and action value. Whereas significant neural signals for action value were found in both structures before animal's choice of action, signals related to the upcoming choice were relatively weak and began to emerge only in the dorsal striatum ϳ200 ms before the behavioral manifestation of the animal's choice. In contrast, once the animal revealed its choice, signals related to choice and its value increased steeply and persisted until the outcome of animal's choice was revealed, so that some neurons in both structures concurrently conveyed signals related to animal's choice, its outcome, and the value of chosen action. Thus, all the components necessary for updating values of chosen actions were available in the striatum. These results suggest that the striatum not only represents values associated with potential choices before animal's choice of action, but might also update the value of chosen action once its outcome is revealed. In contrast, action selection might take place elsewhere or in the dorsal striatum only immediately before its behavioral manifestation.

show abstract

Section: Role Of Striatum In Action Selectionsupporting

confidence: 91%

Section: Functions Of Dorsal Versus Ventral Striatumsupporting

confidence: 58%

Section: Functions Of Dorsal Versus Ventral Striatummentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Role of Striatum in Updating Values of Chosen Actions

Kim

Sul²,

Huh³

et al. 2009

J. Neurosci.

189

304

View full text Add to dashboard Cite

show abstract

“…These findings build on burgeoning evidence implicating the dorsal striatum in the selection of actions leading to reward, and in encoding of stimulus-response or stimulus-response-outcome associations (Lauwereyns et al, 2002;Hikosaka et al, 2006;Balleine et al, 2007). At the point of choice, the mid-caudate is involved in driving choice behavior only when that behavior coheres with the predictions of RL and not alternative, model-based strategies.…”

Section: Discussionmentioning

confidence: 60%

Human Dorsal Striatal Activity during Choice Discriminates Reinforcement Learning Behavior from the Gambler's Fallacy

Jessup¹,

O’Doherty²

2011

J. Neurosci.

View full text Add to dashboard Cite

Reinforcement learning theory has generated substantial interest in neurobiology, particularly because of the resemblance between phasic dopamine and reward prediction errors. Actor-critic theories have been adapted to account for the functions of the striatum, with parts of the dorsal striatum equated to the actor. Here, we specifically test whether the human dorsal striatum-as predicted by an actor-critic instantiation-is used on a trial-to-trial basis at the time of choice to choose in accordance with reinforcement learning theory, as opposed to a competing strategy: the gambler's fallacy. Using a partial-brain functional magnetic resonance imaging scanning protocol focused on the striatum and other ventral brain areas, we found that the dorsal striatum is more active when choosing consistent with reinforcement learning compared with the competing strategy. Moreover, an overlapping area of dorsal striatum along with the ventral striatum was found to be correlated with reward prediction errors at the time of outcome, as predicted by the actor-critic framework. These findings suggest that the same region of dorsal striatum involved in learning stimulus-response associations may contribute to the control of behavior during choice, thereby using those learned associations. Intriguingly, neither reinforcement learning nor the gambler's fallacy conformed to the optimal choice strategy on the specific decision-making task we used. Thus, the dorsal striatum may contribute to the control of behavior according to reinforcement learning even when the prescriptions of such an algorithm are suboptimal in terms of maximizing future rewards.

show abstract

Basal Ganglia and The Regulation of Movement

Turner

2009

Encyclopedia of Life Sciences

View full text Add to dashboard Cite

The basal ganglia (BG) is composed of several heavily interconnected nuclei at the base of the cerebrum. A series of anatomically distinct parallel circuits through the BG receive afferent projections from and project back to cortical regions that mediate skeletomotor, oculomotor, frontal associative, and limbic functions. All circuits share a common intrinsic organization captured largely by the heuristic model of direct and indirect pathways that link BG input stations to outputs. Activation of the direct pathway may facilitate movement whereas the indirect pathway may suppress movement. Neuromodulators such as dopamine have different effects on activity and synaptic plasticity in the two pathways. Clinical disorders that involve the BG are often associated with movement disorders. Although the actual roles of the BG in the control of movement are still actively debated, growing evidence suggests it acts as a reinforcement‐driven tutor for learning automatic behavioural routines. Key concepts The BG is organized as anatomically segregated loop circuits that contribute to the control of movement, cognition and motivation. All BG circuits share a common basic organization. Input projections from cortex and thalamus terminate as excitatory synapses in the striatum and subthalamic nucleus. Output projections tonically inhibit target neurons in thalamic and brainstem nuclei. Direct and indirect pathways connect input and output nuclei of the BG. Activation of the direct pathway facilitates movement whereas activation of the indirect pathway suppresses movement. A variety of neuromodulators, such as dopamine, affect the activity of neurons in the direct and indirect pathways differently. Most BG‐related clinical conditions (e.g. Parkinson disease) arise from imbalanced activation of direct and indirect pathways and involve abnormal discharge patterns in BG output neurons. BG circuits may regulate how an animal allocates time and effort to movements and actions and act as a tutor for new skill learning.

show abstract

Basal Ganglia Orient Eyes to Reward

Cited by 345 publications

References 191 publications

Role of Striatum in Updating Values of Chosen Actions

Role of Striatum in Updating Values of Chosen Actions

Human Dorsal Striatal Activity during Choice Discriminates Reinforcement Learning Behavior from the Gambler's Fallacy

Basal Ganglia and The Regulation of Movement

Contact Info

Product

Resources

About