Summary Orbitofrontal cortex (OFC) has long been known to play an important role in decision making. However, the exact nature of that role has remained elusive. Here we propose a new unifying theory of OFC function. We hypothesize that OFC provides an abstraction of currently available information in the form of a labeling of the current task state, which is used for reinforcement learning elsewhere in the brain. This function is especially critical when task states include unobservable information, for instance, from working memory. We use this framework to explain classic findings in reversal learning, delayed alternation, extinction and devaluation, as well as more recent findings showing the effect of OFC lesions on the firing of dopaminergic neurons in ventral tegmental area (VTA) in rodents performing a reinforcement learning task. In addition, we generate a number of testable experimental predictions that can distinguish our theory from other accounts of OFC function.

Reciprocal connections between the orbitofrontal cortex and the basolateral nucleus of the amygdala may provide a critical circuit for the learning that underlies goal-directed behavior. We examined neural activity in rat orbitofrontal cortex and basolateral amygdala during instrumental learning in an olfactory discrimination task. Neurons in both regions fired selectively during the anticipation of rewarding or aversive outcomes. This selective activity emerged early in training, before the rats had learned reliably to avoid the aversive outcome. The results support the concept that the basolateral amygdala and orbitofrontal cortex cooperate to encode information that may be used to guide goal-directed behavior.

The dopamine system is thought to be involved in making decisions about reward. Here we recorded from the ventral tegmental area in rats learning to choose between differently delayed and sized rewards. As expected, the activity of many putative dopamine neurons reflected reward prediction errors, changing when the value of the reward increased or decreased unexpectedly. During learning, neural responses to reward in these neurons waned and responses to cues that predicted reward emerged. Notably, this cue-evoked activity varied with size and delay. Moreover, when rats were given a choice between two differently valued outcomes, the activity of the neurons initially reflected the more valuable option, even when it was not subsequently selected.Dopamine is thought to be essential for reinforcement learning 1-4 . This hypothesis is rooted in single-unit studies, which show that the firing patterns of dopamine neurons accord well with a signal that reports errors in reward prediction 5-15 . This is true of firing for rewards as well as for cues that come to predict rewards. This signal is thought to regulate learning and to guide decision-making; however, few studies have investigated this issue in behavioral tasks that involve active decision-making 11,14 .To address this issue, we recorded single-unit activity in the ventral tegmental area (VTA) of rats performing a variant of a time-discounting task. It is known that animals prefer immediate over delayed reward (time discounting) 16-25 , but the role of dopamine in this impulsive behavior remains unclear 26-30 . We designed a task to isolate changes in neural activity related to reward size from changes in neural activity related to time to reward by independently varying when (after a short or long delay) or what (big or small) reward was to be delivered at the end of the trial. As the rats learned to respond to the more valuable odor cue, many dopamine neurons developed cue-selective activity during odor sampling, firing more strongly for cues that predicted either the more immediate reward or the larger one. Notably, these same neurons exhibited activity that was consistent with prediction errors during unexpected delivery or omission of reward. When the rats were given the opportunity to choose between two

Clinical evidence indicates that damage to ventromedial prefrontal cortex disrupts goal-directed actions that are guided by motivational and emotional factors. As a consequence, patients with such damage characteristically engage in maladaptive behaviors. Other research has shown that neurons in the corresponding orbital region of prefrontal cortex in laboratory animals encode information regarding the incentive properties of goals or expected events. The present study investigates the effect of neurotoxic orbitofrontal cortex (OFC) lesions in the rat on responses that are normally influenced by associations between a conditioned stimulus (CS) and the incentive value of reinforcement. Rats were first trained to associate a visual CS with delivery of food pellets to a food cup. As a consequence of learning, rats approached the food cup during the CS in anticipation of reinforcement. In a second training phase, injection of LiCl followed consumption of the food unconditioned stimulus (US) in the home cage, a procedure used to alter the incentive value of the US. Subsequently, rats were returned to the conditioning chamber, and their responding to the CS in the absence of the food US was tested. Lesions of OFC did not affect either the initial acquisition of a conditioned response to the light CS in the first training phase or taste aversion learning in the second training phase. In the test for devaluation, however, OFC rats exhibited no change in conditioned responding to the visual CS. This outcome contrasts with the behavior of control rats; after devaluation of the US a significant decrease occurred in approach to the food cup during presentation of the CS. The results reveal an inability of a cue to access representational information about the incentive value of associated reinforcement after OFC damage.

The orbitofrontal cortex (OFC) is crucial for changing established behaviour in the face of unexpected outcomes. This function has been attributed to the role of the OFC in response inhibition or to the idea that the OFC is a rapidly flexible associative-learning area. However, recent data contradict these accounts, and instead suggest that the OFC is crucial for signalling outcome expectancies. We suggest that this function -signalling of expected outcomes -can also explain the crucial role of the OFC in changing behaviour in the face of unexpected outcomes.The orbitofrontal cortex (OFC) has long been associated with adaptive, flexible behaviour in the face of changing contingencies and unexpected outcomes. In 1868, John Harlow 1 described the erratic, inflexible, stimulus-bound behaviour of Phineas Gage, who reportedly suffered extensive damage to the orbital and midline prefrontal regions 2 . Since then, owing to increasingly refined experimental work, the regulation of flexible behaviour has been attributed to the OFC, a set of loosely defined areas in the prefrontal regions that overlie the orbits.In this Perspective, we first review data showing that the OFC is crucial for changing behaviour in the face of unexpected outcomes. Then we describe the two dominant hypotheses that have been advanced to explain this function, and follow this with data that directly contradict both of these. We focus on a region of the OFC for which there are strong anatomical and functional parallels across rodents and primates; this area encompasses the dorsal bank of the rhinal sulcus in rats, including lateral orbital regions and anterior parts of the agranular insular cortex. These cortical fields have a pattern of connectivity with the rostral basolateral amygdala, the ventral striatum, the mediodorsal thalamus and sensory regions that is qualitatively similar to the pattern of connectivity of areas 11, 12 and 13 in the primate orbital prefrontal cortex 3,4 . Furthermore, as we outline below, neurophysiological and lesion studies show remarkable similarities in the functions of these areas across species. These studies indicate that this part of the OFC is crucial for signalling information about expected outcomes and for using that Correspondence NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author Manuscript information to guide behaviour. We suggest that signals from this part of the OFC that relate to expected outcomes also contribute to the detection of errors in reward prediction when contingencies are changing, thereby facilitating changes in associative representations in other brain areas and, ultimately, behaviour. The OFC is crucial for flexible behaviourThe role of the OFC in flexible behaviour is typically assessed using tasks in which subjects must change an established behavioural response in order to adapt to new contingencies. FIGURE 1 shows an example in rats performing a simple odour discrimination task. Here, rats are trained to sample odour cues at a centrally located port and then ...

Orbitofrontal cortex (OFC) is part of a network of structures involved in adaptive behavior and decision making. Interconnections between OFC and basolateral amygdala (ABL) may be critical for encoding the motivational significance of stimuli used to guide behavior. Indeed, much research indicates that neurons in OFC and ABL fire selectively to cues based on their associative significance. In the current study recordings were made in each region within a behavioral paradigm that allowed comparison of the development of associative encoding over the course of learning. In each recording session, rats were presented with novel odors that were informative about the outcome of making a response and had to learn to withhold a response after sampling an odor that signaled a negative outcome. In some cases, reversal training was performed in the same session as the initial learning. Ninety-six of the 328 neurons recorded in OFC and 60 of the 229 neurons recorded in ABL exhibited selective activity during evaluation of the odor cues after learning had occurred. A substantial proportion of those neurons in ABL developed selective activity very early in training, and many reversed selectivity rapidly after reversal. In contrast, those neurons in OFC rarely exhibited selective activity during odor evaluation before the rats reached the criterion for learning, and far fewer reversed selectivity after reversal. The findings support a model in which ABL encodes the motivational significance of cues and OFC uses this information in the selection and execution of an appropriate behavioral strategy.

Although the human amygdala and striatum have both been implicated in associative learning, only the striatum’s contribution has been consistently computationally characterized. Using a reversal learning task, we demonstrate that amygdala BOLD activity tracks associability as estimated by a computational model, and dissociates it from the striatal representation of reinforcement prediction error. These results extend the computational learning approach from striatum to amygdala, demonstrating their complementary roles in aversive learning.

The orbitofrontal cortex (OFC) and basolateral amygdala (BLA) are critical for using learned representations of outcomes to guide behavior. Neurophysiological findings suggest complementary roles in which the BLA acquires associations between cues and outcomes and the OFC subsequently uses them to guide behavior. Here, we have used a reinforcer devaluation paradigm to test this hypothesis. In this paradigm, rats are first trained to associate a light conditioned stimulus (CS) with a food outcome, and then the food is devalued by pairing it with illness. After this devaluation procedure, responding to the CS is assessed in a single probe session. Previously, we have shown that BLA and OFC lesions made before training do not affect the acquisition of conditioned responding but do impair the sensitivity of that responding to reinforcer devaluation. Rats with such lesions fail to exhibit the spontaneous decrease in conditioned responding to the light cue observed in controls in the probe test. Here, we have extended those findings by showing that performance in the probe test is impaired by OFC lesions made after light-food conditioning but not by BLA lesions made after that training. These findings indicate that the OFC and BLA play different roles in mediating normal goal-directed performance in this, and likely other, settings. The BLA seems critical to forming representations linking cues to the incentive properties of outcomes but not for maintaining these representations in memory, updating them with new information, or for expressing them in behavior. In contrast, the OFC seems essential for one or more of these latter processes.