SummaryDecision making is often considered to arise out of contributions from a model-free habitual system and a model-based goal-directed system. Here, we investigated the effect of a dopamine manipulation on the degree to which either system contributes to instrumental behavior in a two-stage Markov decision task, which has been shown to discriminate model-free from model-based control. We found increased dopamine levels promote model-based over model-free choice.

Investigations of the underlying mechanisms of choice in humans have focused on learning from prediction errors, and so the computational structure of value based planning is comparatively underexplored. Using behavioural and neuroimaging analyses of a minimax decision task, we show that the computational processes underlying forward planning are expressed in the anterior caudate nucleus as values of individual branching steps in a decision tree. In contrast, values represented in the putamen pertain solely to values learnt during extensive training. During actual choice, both striatal areas show a functional coupling to ventromedial prefrontal cortex, consistent with this region acting as a value comparator. Our findings point towards an architecture of choice in which segregated value systems operate in parallel in the striatum for planning and extensively trained choices, with medial prefrontal cortex integrating their outputs.

Action-based decision making involves choices between different physical actions to obtain rewards. To make such decisions the brain needs to assign a value to each action and then compare them to make a choice. Using fMRI in human subjects, we found evidence for action-value signals in supplementary motor cortex. Separate brain regions, most prominently ventromedial prefrontal cortex, were involved in encoding the expected value of the action that was ultimately taken. These findings differentiate two main forms of value signals in the human brain: those relating to the value of each available action, likely reflecting signals that are a precursor of choice, and those corresponding to the expected value of the action that is subsequently chosen, and therefore reflecting the consequence of the decision process. Furthermore, we also found signals in the dorsomedial frontal cortex that resemble the output of a decision comparator, which implicates this region in the computation of the decision itself.acc ͉ action value ͉ reinforcement learning ͉ sma ͉ vmpfc C onsider a goalkeeper trying to stop a soccer ball during a penalty kick. Within a brief amount of time he needs to choose between jumping to the left or right goal posts. Repeated play against the same opponents allows him to learn about their scoring tendencies, which can be used to compute the values of a left and a right jump before making a decision. It is a long-established view in economics, psychology, and computational neuroscience that the brain makes choices among actions by first computing a value for each possible action, and then selecting one of them on the basis of those values (1-3). This raises two fundamental questions in decision neuroscience: (1) where in the brain are the values of different types of actions encoded? and (2) how and where does the brain compare those values to generate a choice?An emerging theme in decision neuroscience is that organisms need to make a number of value-related computations to make even simple choices (4). Consider the case of action-based choice exemplified by the goalkeeper's problem. First, he needs to assign a value to each action under consideration. These signals, known as action values, encode the value of each action before choice and regardless of whether it is subsequently chosen or not, which allows them to serve as inputs into the decision-making process (5-7). Second, these action values are compared to generate a choice.