Action and Outcome Encoding in the Primate Caudate Nucleus

Lau, Brian; Glimcher, Paul W.

doi:10.1523/jneurosci.3060-07.2007

Cited by 159 publications

(160 citation statements)

References 71 publications

(89 reference statements)

Supporting

Mentioning

138

Contrasting

Order By: Relevance

“…By contrast, the parsimonious alternative pro-posed here, whereby in such contexts a decision is made between the stimuli, would not suffer from the same scaling problem. We did not find action value signals (5,12,28) in the SC or AC conditions even at liberal thresholds. A possible explanation for the lack of action value signals could be that, in contrast to previous paradigms, in the current study subjects did not have to keep track of action values over trials and doing so would have not improved performance.…”

Section: Discussioncontrasting

confidence: 83%

Economic choices can be made using only stimulus values

Wunderlich

Rangel

O’Doherty

2010

Proc. Natl. Acad. Sci. U.S.A.

120

View full text Add to dashboard Cite

Decision-making often involves choices between different stimuli, each of which is associated with a different physical action. A growing consensus suggests that the brain makes such decisions by assigning a value to each available option and then comparing them to make a choice. An open question in decision neuroscience is whether the brain computes these choices by comparing the values of stimuli directly in goods space or instead by first assigning values to the associated actions and then making a choice over actions. We used a functional MRI paradigm in which human subjects made choices between different stimuli with and without knowledge of the actions required to obtain the different stimuli. We found neural correlates of the value of the chosen stimulus (a postdecision signal) in ventromedial prefrontal cortex before the actual stimulus-action pairing was revealed. These findings provide support for the hypothesis that the brain is capable of making choices in the space of goods without first transferring values into action space.decision-making | functional MRI | goods space | reinforcement learning | ventromedial prefrontal cortex I magine you are thirsty and walk up to a vending machine that is serving a variety of soft drink beverages. On the machine you see the brand marks of the offered beverages, and because you had previously sampled them, you easily assign values to each drink based on their taste. To get the desired beverage, you press the button that is distinctively associated with the preferred option. This situation exemplifies many of the decisions that humans and animals make in daily life. It is a well-established belief among economists, psychologists, and neuroscientists that the brain solves such choice problems by first computing a value for each alternative and then selecting the one that has the highest value (1-4). Neuroscientists have considered two possible alternative ways for how values might be compared in order to make a choice in these situations.First is the actions-based model, in which choices are embedded in premotor processes of action selection: the values of stimuli are passed as action values to the motor plans required to obtain them, and the decision is then made in action space by comparing action values (5-7). Values are learned through experience and associated with each motor plan, and during choice a single motor act eventually emerges through a winner-takes-all process. Although this action based model of making decisions may seem convoluted, it is in fact the predominant view among many decision neuroscientists who have found value signals in areas of the brain known to be involved in representing and planning movements such as lateral parietal and premotor cortices (8-11). Further evidence for the view that decisions are computed by a comparison between actions comes from the finding of action value signals in several regions of the brain, including the caudate nucleus (5, 6), supplementary motor cortex (12), and action-related value signals in lateral int...

show abstract

Section: Discussioncontrasting

confidence: 83%

Economic choices can be made using only stimulus values

Wunderlich

Rangel

O’Doherty

2010

Proc. Natl. Acad. Sci. U.S.A.

120

View full text Add to dashboard Cite

show abstract

“…Although these signals are precursors of choice, they are not instances of action values since they are stimulus-based and independent of the action required to obtain them. To date, only three monkey electrophysiology studies have found evidence for the presence of action-value signals for hand and eye movements in the striatum during simple decisionmaking tasks (5)(6)(7). This study extends their findings in three directions.…”

supporting

confidence: 76%

“…First, he needs to assign a value to each action under consideration. These signals, known as action values, encode the value of each action before choice and regardless of whether it is subsequently chosen or not, which allows them to serve as inputs into the decision-making process (5)(6)(7). Second, these action values are compared to generate a choice.…”

mentioning

confidence: 99%

Neural computations underlying action-based decision making in the human brain

Wunderlich

Rangel

O’Doherty

2009

Proc. Natl. Acad. Sci. U.S.A.

267

214

View full text Add to dashboard Cite

Action-based decision making involves choices between different physical actions to obtain rewards. To make such decisions the brain needs to assign a value to each action and then compare them to make a choice. Using fMRI in human subjects, we found evidence for action-value signals in supplementary motor cortex. Separate brain regions, most prominently ventromedial prefrontal cortex, were involved in encoding the expected value of the action that was ultimately taken. These findings differentiate two main forms of value signals in the human brain: those relating to the value of each available action, likely reflecting signals that are a precursor of choice, and those corresponding to the expected value of the action that is subsequently chosen, and therefore reflecting the consequence of the decision process. Furthermore, we also found signals in the dorsomedial frontal cortex that resemble the output of a decision comparator, which implicates this region in the computation of the decision itself.acc ͉ action value ͉ reinforcement learning ͉ sma ͉ vmpfc C onsider a goalkeeper trying to stop a soccer ball during a penalty kick. Within a brief amount of time he needs to choose between jumping to the left or right goal posts. Repeated play against the same opponents allows him to learn about their scoring tendencies, which can be used to compute the values of a left and a right jump before making a decision. It is a long-established view in economics, psychology, and computational neuroscience that the brain makes choices among actions by first computing a value for each possible action, and then selecting one of them on the basis of those values (1-3). This raises two fundamental questions in decision neuroscience: (1) where in the brain are the values of different types of actions encoded? and (2) how and where does the brain compare those values to generate a choice?An emerging theme in decision neuroscience is that organisms need to make a number of value-related computations to make even simple choices (4). Consider the case of action-based choice exemplified by the goalkeeper's problem. First, he needs to assign a value to each action under consideration. These signals, known as action values, encode the value of each action before choice and regardless of whether it is subsequently chosen or not, which allows them to serve as inputs into the decision-making process (5-7). Second, these action values are compared to generate a choice.

show abstract

“…However, reasonable candidates include dopaminergic (DA) neurons in the ventral tegemental area (VTA) and the substantia nigra pars compacta (SNc) that might influence activity in visual cortex via direct projections to early areas of visual cortex (Berger et al 1988(Berger et al , 1991Devoto and Flore 2006). However, these projections are generally thought to be sparse, so it is likely that indirect DA signals relayed through the striatum and then to frontal and parietal cortex play an important role in regulating value-related changes in early visual cortex (Barraclough et al 2004;Ding and Hikosaka 2006;Dorris and Glimcher 2004;Gläscher et al 2009;Glimcher 2003;Hikosaka et al 2008;Hollerman and Schultz 1998;Ikeda and Hikosaka 2003;Lau and Glimcher 2007;Leon and Shadlen 1999;Luk and Wallis 2009;Platt and Glimcher 1999;Schultz and Dickinson 2000;Seo et al 2007;Sugrue et al 2004;Wallis and Miller 2003;Watanabe 1996). Indeed, many of the cortical targets of reward signals-such as oculomotor neurons in frontal and parietal cortex-are ideally situated to send modulatory feedback signals to earlier sensory areas so that the cortical representation of high-value stimulus features can be enhanced (Bisley and Goldberg 2003;Ding and Hikosaka 2006;Gold and Shadlen 2007;Serences and Yantis 2006;Shadlen and Newsome 2001).…”

Section: Value and Population Responses In Human Visual Cortexmentioning

confidence: 99%

Population Response Profiles in Early Visual Cortex Are Biased in Favor of More Valuable Stimuli

Serences

Saproo

2010

Journal of Neurophysiology

104

View full text Add to dashboard Cite

Serences JT, Saproo S. Population response profiles in early visual cortex are biased in favor of more valuable stimuli. J Neurophysiol 104: 76 -87, 2010. First published April 21, 2010 doi:10.1152/jn.01090.2009. Voluntary and stimulus-driven shifts of attention can modulate the representation of behaviorally relevant stimuli in early areas of visual cortex. In turn, attended items are processed faster and more accurately, facilitating the selection of appropriate behavioral responses. Information processing is also strongly influenced by past experience and recent studies indicate that the learned value of a stimulus can influence relatively late stages of decision making such as the process of selecting a motor response. However, the learned value of a stimulus can also influence the magnitude of cortical responses in early sensory areas such as V1 and S1. These early effects of stimulus value are presumed to improve the quality of sensory representations; however, the nature of these modulations is not clear. They could reflect nonspecific changes in response amplitude associated with changes in general arousal or they could reflect a bias in population responses so that high-value features are represented more robustly. To examine this issue, subjects performed a two-alternative forced choice paradigm with a variableinterval payoff schedule to dynamically manipulate the relative value of two stimuli defined by their orientation (one was rotated clockwise from vertical, the other counterclockwise). Activation levels in visual cortex were monitored using functional MRI and feature-selective voxel tuning functions while subjects performed the behavioral task. The results suggest that value not only modulates the relative amplitude of responses in early areas of human visual cortex, but also sharpens the response profile across the populations of feature-selective neurons that encode the critical stimulus feature (orientation). Moreover, changes in space-or feature-based attention cannot easily explain the results because representations of both the selected and the unselected stimuli underwent a similar feature-selective modulation. This sharpening in the population response profile could theoretically improve the probability of correctly discriminating high-value stimuli from low-value alternatives.

show abstract

Action and Outcome Encoding in the Primate Caudate Nucleus

Cited by 159 publications

References 71 publications

Economic choices can be made using only stimulus values

Economic choices can be made using only stimulus values

Neural computations underlying action-based decision making in the human brain

Population Response Profiles in Early Visual Cortex Are Biased in Favor of More Valuable Stimuli

Contact Info

Product

Resources

About