Subjects with schizophrenia are impaired at reinforcement-driven reversal learning from as early as their first episode. The neurobiological basis of this deficit is unknown. We obtained behavioral and fMRI data in 24 unmedicated, primarily first episode, schizophrenia patients and 24 age-, IQ- and gender-matched healthy controls during a reversal learning task. We supplemented our fMRI analysis, focusing on learning from prediction errors, with detailed computational modeling to probe task solving strategy including an ability to deploy an internal goal directed model of the task. Patients displayed reduced functional activation in the ventral striatum (VS) elicited by prediction errors. However, modeling task performance revealed that a subgroup did not adjust their behavior according to an accurate internal model of the task structure, and these were also the more severely psychotic patients. In patients who could adapt their behavior, as well as in controls, task solving was best described by cognitive strategies according to a Hidden Markov Model. When we compared patients and controls who acted according to this strategy, patients still displayed a significant reduction in VS activation elicited by informative errors that precede salient changes of behavior (reversals). Thus, our study shows that VS dysfunction in schizophrenia patients during reward-related reversal learning remains a core deficit even when controlling for task solving strategies. This result highlights VS dysfunction is tightly linked to a reward-related reversal learning deficit in early, unmedicated schizophrenia patients.

Dual system theories suggest that behavioral control is parsed between a deliberative "model-based" and a more reflexive "modelfree" system. A balance of control exerted by these systems is thought to be related to dopamine neurotransmission. However, in the absence of direct measures of human dopamine, it remains unknown whether this reflects a quantitative relation with dopamine either in the striatum or other brain areas. Using a sequential decision task performed during functional magnetic resonance imaging, combined with striatal measures of dopamine using [ 18 F]DOPA positron emission tomography, we show that higher presynaptic ventral striatal dopamine levels were associated with a behavioral bias toward more model-based control. Higher presynaptic dopamine in ventral striatum was associated with greater coding of model-based signatures in lateral prefrontal cortex and diminished coding of model-free prediction errors in ventral striatum. Thus, interindividual variability in ventral striatal presynaptic dopamine reflects a balance in the behavioral expression and the neural signatures of model-free and model-based control. Our data provide a novel perspective on how alterations in presynaptic dopamine levels might be accompanied by a disruption of behavioral control as observed in aging or neuropsychiatric diseases such as schizophrenia and addiction.dopamine | decision making | reinforcement learning | PET | fMRI H uman choice behavior is influenced by both habitual and goal-directed systems (1). For example, having enjoyed a delicious dinner makes another subsequent visit to the same restaurant more likely. Upon returning at a later point, another visit could happen reflexively when walking past the restaurant, or alternatively be planned and involve reflection, for instance, by checking recent customer reviews to bolster against possible changes. These two decision modes differ fundamentally in terms of their control over actions and associated outcome consequences. Reflexive habitual preferences are retrospective and arise from a slow accumulation of rewards via iterative updating of expectations (2), for example by repeating dinner at the same place after having previously enjoyed tasty food there. In contrast, goal-directed behavior requires a prospective consideration of future outcomes associated with a set of actions (3). For example, knowledge that the chef has changed and subsequent reviews have been less good should reduce one's expectations. Thus, in the face of such change, a goaldirected system can adapt quickly, whereas a habitual system needs to experience an actual outcome before it can alter behavior in an adaptive manner (4). This dual-system theory has been formalized within computational models of learning that update expectations based on past rewards ("model-free") or map possible actions to their potential outcomes ("model-based") (5). There is evidence that model-based learning signals during the acquisition of task structure are encoded within prefrontal-parietal cortices, where...