Habits are characterized by an insensitivity to their consequences and, as such, can be distinguished from goal-directed actions. The neural basis of the development of demonstrably outcome-insensitive habitual actions in humans has not been previously characterized. In this experiment, we show that extensive training on a free-operant task reduces the sensitivity of participants' behavior to a reduction in outcome value. Analysis of functional magnetic resonance imaging data acquired during training revealed a significant increase in task-related cue sensitivity in a right posterior putamen-globus pallidus region as training progressed. These results provide evidence for a shift from goal-directed to habit-based control of instrumental actions in humans, and suggest that cue-driven activation in a specific region of dorsolateral posterior putamen may contribute to the habitual control of behavior in humans.

Research has increasingly implicated the striatum in the processing of reward-related information in both animals and humans. However, it is unclear whether human striatal activation is driven solely by the hedonic properties of rewards or whether such activation is reliant on other factors, such as anticipation of upcoming reward or performance of an action to earn a reward. We used event-related functional magnetic resonance imaging to investigate hemodynamic responses to monetary rewards and punishments in three experiments that made use of an oddball paradigm. We presented reward and punishment displays randomly in time, following an anticipatory cue, or following a button press response. Robust and differential activation of the caudate nucleus occurred only when a perception of contingency existed between the button press response and the outcome. This finding suggests that the caudate is involved in reinforcement of action potentially leading to reward, rather than in processing reward per se.

A popular hypothesis in the social sciences is that humans have social preferences to reduce inequality in outcome distributions because it has a negative impact on their experienced reward. Although there is a large body of behavioural and anthropological evidence consistent with the predictions of these theories, there is no direct neural evidence for the existence of inequality-averse preferences. Such evidence would be especially useful because some behaviours that are consistent with a dislike for unequal outcomes could also be explained by concerns for social image or reciprocity, which do not require a direct aversion towards inequality. Here we use functional MRI to test directly for the existence of inequality-averse social preferences in the human brain. Inequality was created by recruiting pairs of subjects and giving one of them a large monetary endowment. While both subjects evaluated further monetary transfers from the experimenter to themselves and to the other participant, we measured neural responses in the ventral striatum and ventromedial prefrontal cortex, two areas that have been shown to be involved in the valuation of monetary and primary rewards in both social and non-social contexts. Consistent with inequality-averse models of social preferences, we find that activity in these areas was more responsive to transfers to others than to self in the 'high-pay' subject, whereas the activity of the 'low-pay' subject showed the opposite pattern. These results provide direct evidence for the validity of this class of models, and also show that the brain's reward circuitry is sensitive to both advantageous and disadvantageous inequality.

Adults have difficulty discriminating nonnative phonetic contrasts, but under certain circumstances training can lead to improvement in this ability. Despite the ubiquitous use of performance feedback in training paradigms in this and many other domains, the mechanisms by which feedback affects learning are not well understood. In this event-related functional magnetic resonance imaging study, we examined how performance feedback is processed during perceptual learning. Thirteen Japanese speakers for whom the English phonemes [r] and [l] were nondistinct performed an identification task of the words "road" and "load" that has been shown to be effective in inducing learning only when performance feedback is present. Each subject performed alternating runs of training with and without feedback, followed by performance of a card-guessing task with monetary reward and punishment outcomes. We found that the caudate nucleus was more robustly activated bilaterally when performing the perceptual identification task with feedback than without feedback, and the right caudate nucleus also showed a differential response to positive and negative feedback. Moreover, using a within-subjects design, we found that the caudate nucleus also showed a similar activation pattern to monetary reward and punishment outcomes in the card-guessing task. These results demonstrate that the caudate responds to positive and negative feedback during learning in a manner analogous to its processing of extrinsic affective reinforcers and indicate that this region may be a critical moderator of the influence of feedback on learning. These findings impact our broader understanding of the mechanisms underlying nondeclarative learning and language acquisition.

Contingency theories of goal-directed action propose that experienced disjunctions between an action and its specific consequences, as well as conjunctions between these events, contribute to encoding the action-outcome association. Although considerable behavioral research in rats and humans has provided evidence for this proposal, relatively little is known about the neural processes that contribute to the two components of the contingency calculation. Specifically, while recent findings suggest that the influence of action-outcome conjunctions on goal-directed learning is mediated by a circuit involving ventromedial prefrontal, medial orbitofrontal cortex, and dorsomedial striatum, the neural processes that mediate the influence of experienced disjunctions between these events are unknown. Here we show differential responses to probabilities of conjunctive and disjunctive reward deliveries in the ventromedial prefrontal cortex, the dorsomedial striatum, and the inferior frontal gyrus. Importantly, activity in the inferior parietal lobule and the left middle frontal gyrus varied with a formal integration of the two reward probabilities, ⌬P, as did response rates and explicit judgments of the causal efficacy of the action.

The striatum has been shown to be a key region in the processing of reward-related information. The head of the caudate nucleus has been implicated in processing performance feedback, or in other words, information about the outcomes of one's actions. However, feedback provides multiple types of information, and it is not clear which of these types of information drive a caudate response. We sought to determine whether the signal in the caudate differed when feedback was informative but only arbitrarily related to performance versus when it provided information about goal achievement. To do this, we used functional magnetic resonance imaging (fMRI) to examine caudate activation during a feedback-based paired associate word learning task. During an initial round of 60 distinct trials, participants chose one of two responses on each trial and received feedback about whether their responses were correct. On the subsequent two rounds, the 60 trials were repeated and participants chose their responses based on their memory of the correct answer. The caudate nuclei were strongly engaged only during the second two rounds, when feedback reflected the accuracy of memory. These results support the idea that feedback-based caudate activation is context dependent: the caudate can be engaged in feedback-based declarative memory tasks, but it is more strongly engaged when feedback is "earned" by performance than when it is informative but not tied to goal achievement.Keywords basal ganglia; fMRI; learning; memory; reinforcement; striatum The head of the caudate nucleus, which lies in the dorsomedial portion of the striatum, has been implicated in processing reward-related information (Delgado et al., 2000;Elliott et al., 2000b;Knutson et al., 2000), including the processing of performance-related feedback (e.g., (Elliott et al., 1997;Poldrack et al., 2001;Tricomi et al., 2006). However, little is known about the specific role that the caudate plays in feedback-based learning. The goal of this experiment was to examine the parameters that govern feedback-related caudate activation.Neuropsychological work has implicated the striatum in habit learning; both Parkinson's disease and Huntington's disease reflect disruptions in striatal function, and in both diseases performance is impaired on gradually learned tasks in which there is little awareness of what Corresponding author: Elizabeth Tricomi, Division of the Humanities and Social Sciences, MC 228-77, California Institute of Technology, Pasadena, California 91125, Telephone: (626) Fax: (626) 405-9841, Electronic mail: etricomi@caltech.edu. Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the conten...

Delay discounting refers to the reduction of the value of a future reward as the delay to that reward increases. The rate at which individuals discount future rewards varies as a function of both individual and contextual differences, and high delay discounting rates have been linked with problematic behaviors, including drug abuse and gambling. The current study investigated the effects of acute anticipatory stress on delay discounting, while considering two important factors: individual perceptions of stress and whether the stressful situation is future-focused or present-focused. Half of the participants experienced acute stress by anticipating giving a videotaped speech. This stress was either future-oriented (speech about future job) or present-oriented (speech about physical appearance). They then performed a delay discounting task, in which they chose between smaller, immediate rewards, and larger, delayed rewards. Their scores on the Perceived Stress Scale were also collected. The way in which one appraises stressful situations interacts with acute stress to influence choices; under stressful conditions, delay discounting rate was highest in individuals with low trait perceived stress and lowest for individuals with high trait perceived stress. This result might be related to individual variation in reward responsiveness under stress. Furthermore, the time orientation of the task interacted with its stressfulness to affect the individual’s propensity to choose immediate rewards. These findings add to our understanding of the intermediary factors between stress and decision-making.

Learning commonly requires feedback about the consequences of one’s actions, which can drive learners to modify their behavior. Motivation may determine how sensitive an individual might be to such feedback, particularly in educational contexts where some students value academic achievement more than others. Thus, motivation for a task might influence the value placed on performance feedback and how effectively it is used to improve learning. To investigate the interplay between intrinsic motivation and feedback processing, we used functional magnetic resonance imaging (fMRI) during feedback-based learning before and after a novel manipulation based on motivational interviewing, a technique for enhancing treatment motivation in mental health settings. Because of its role in the reinforcement learning system, the striatum is situated to play a significant role in the modulation of learning based on motivation. Consistent with this idea, motivation levels during the task were associated with sensitivity to positive versus negative feedback in the striatum. Additionally, heightened motivation following a brief motivational interview was associated with increases in feedback sensitivity in the left medial temporal lobe. Our results suggest that motivation modulates neural responses to performance-related feedback, and furthermore that changes in motivation facilitates processing in areas that support learning and memory.