Basal Ganglia Circuits for Reward Value–Guided Behavior

Abstract

The basal ganglia are equipped with inhibitory and disinhibitory mechanisms that enable to choose valuable objects and actions. Notably, a value can be determined flexibly by recent experience or stably by prolonged experience. Recent studies have revealed that the head and tail of the caudate nucleus selectively and differentially process flexible and stable values of visual objects. These signals are sent to the superior colliculus through different parts of the substantia nigra, so that the animal looks preferentially at high-valued objects, but in different manners. Relying on short-term value memories, the caudate head circuit allows gaze to move expectantly to recently valued objects. Relying on long-term value memories, the caudate tail circuit allows gaze to move automatically to previously valued objects. The basal ganglia also contain an equivalent parallel mechanism for action values. Such flexible-stable parallel mechanisms for object and action values create a highly adaptable system for decision making.

New is always better: Novelty modulates oculomotor learning

“…Dopamine neurons become active when an outcome is better than expected, thereby unifying both the surprise and the reward components of novelty (Schultz, 1998). The basal ganglia, with their dopaminergic signaling, are involved in eye-movement control as well as rewardbased learning, and modify eye-movement vigor to rewarding stimuli (Hikosaka, Kim, Yasuda, & Yamamoto, 2014;Wittmann et al, 2007). An involvement of the basal ganglia in saccadic adaptation has not been researched systematically, but evidence from an individual with Parkinson's disease suggests a possible contribution (MacAskill, Anderson, & Jones, 2002).…”

The influence of image content on oculomotor plasticity

“…The influence of the content of the image, and perhaps its motivational or rewarding character, may be conveyed by this route or, alternatively, by direct influences on cerebellar plasticity. A further possibility is the involvement of basal ganglia circuits that contribute to eye movement control and to reward-based learning (Hikosaka, Kim, Yasuda, & Yamamoto, 2014). The basal ganglia maintain long-term associations between stimuli and rewards and modify eye movement vigor to rewarding stimuli.…”

Reward modulates oculomotor competition between differently valued stimuli

“…In contrast, we hypothesized that landing positions in the post-training phase would not show the typical global effect, but instead show a bias towards the object that was associated with a higher reward value during the reward-training phase. Congruent with the monkey literature on automatic reward effects (see Hikosaka et al, 2014), higher compared with lower reward value associated objects will evoke a larger peak of activity in the saccade map in the post-training phase, resulting in a bias towards the objects with a higher reward value.…”

“…Evidence from monkey studies indicates that the caudate nucleus (CD) and the substantia nigra pars reticulata (SNr) play a key role in reward dependent changes in activity in the saccade map (Hikosaka et al, 2014;Yamamoto, Kim, & Hikosaka, 2013;Yasuda, Yamamoto, & Hikosaka, 2012). Cortical inputs carrying spatial information and dopaminergic signals carrying reward information are thought to be integrated in the basal ganglia where saccade related reward modulation takes place.…”