Deciding when to leave a depleting resource to exploit another is a fundamental problem for all decision makers. The neuronal mechanisms mediating patch-leaving decisions remain unknown. We found that neurons in primate (Macaca mulatta) dorsal anterior cingulate cortex, an area that is linked to reward monitoring and executive control, encode a decision variable signaling the relative value of leaving a depleting resource for a new one. Neurons fired during each sequential decision to stay in a patch and, for each travel time, these responses reached a fixed threshold for patch-leaving. Longer travel times reduced the gain of neural responses for choosing to stay in a patch and increased the firing rate threshold mandating patch-leaving. These modulations more closely matched behavioral decisions than any single task variable. These findings portend an understanding of the neural basis of foraging decisions and endorse the unification of theoretical and experimental work in ecology and neuroscience.

SUMMARY Curiosity is a basic element of our cognition, yet its biological function, mechanisms, and neural underpinning remain poorly understood. It is nonetheless a motivator for learning, influential in decision-making, and crucial for healthy development. One factor limiting our understanding of it is the lack of a widely agreed upon delineation of what is and is not curiosity; another factor is the dearth of standardized laboratory tasks that manipulate curiosity in the lab. Despite these barriers, recent years have seen a major growth of interest in both the neuroscience and psychology of curiosity. In this Perspective, we advocate for the importance of the field, provide a selective overview of its current state, and describe tasks that are used to study curiosity and information-seeking. We propose that, rather than worry about defining curiosity, it is more helpful to consider the motivations for information-seeking behavior and to study it in its ethological context.

Recent theories suggest that reward-based choice reflects competition between value signals in the ventromedial prefrontal cortex (vmPFC). We tested this idea by recording vmPFC neurons while macaques performed a gambling task with asynchronous offer presentation. We found that neuronal activity shows four patterns consistent with selection via mutual inhibition. (1) Correlated tuning for probability and reward size, suggesting that vmPFC carries an integrated value signal, (2) anti-correlated tuning curves for the two options, suggesting mutual inhibition, (3) neurons rapidly come to signal the value of the chosen offer, suggesting the circuit serves to produce a choice, (4) after regressing out the effects of option values, firing rates still could predict choice – a choice probability signal. In addition, neurons signaled gamble outcomes, suggesting that vmPFC contributes to both monitoring and choice processes. These data suggest a possible mechanism for reward-based choice and endorse the centrality of vmPFC in that process.

In attentional models of learning, associations between actions and subsequent rewards are stronger when outcomes are surprising, regardless of their valence. Despite the behavioral evidence that surprising outcomes drive learning, neural correlates of unsigned reward prediction errors remain elusive. Here we show that in a probabilistic choice task, trial-to-trial variations in preference track outcome surprisingness. Concordant with this behavioral pattern, responses of neurons in macaque (Macaca mulatta) dorsal anterior cingulate cortex (dACC) to both large and small rewards were enhanced when the outcome was surprising. Moreover, when, on some trials, probabilities were hidden, neuronal responses to rewards were reduced, consistent with the idea that the absence of clear expectations diminishes surprise. These patterns are inconsistent with the idea that dACC neurons track signed errors in reward prediction, as dopamine neurons do. Our results also indicate that dACC neurons do not signal conflict. In the context of other studies of dACC function, these results suggest a link between reward-related modulations in dACC activity and attention and motor control processes involved in behavioral adjustment. More speculatively, these data point to a harmonious integration between reward and learning accounts of ACC function on one hand, and attention and cognitive control accounts on the other.

When has the world changed enough to warrant a new approach? The answer depends upon current needs, behavioral flexibility, and prior knowledge about the environment. Formal approaches solve the problem by integrating the recent history of rewards, errors, uncertainty, and context via Bayesian inference to detect changes in the world and alter behavioral policy. Neuronal activity in posterior cingulate cortex (CGp)-a key node in the default network-is known to vary with learning, memory, reward, and task engagement. We propose that these modulations reflect the underlying process of change detection and motivate subsequent shifts in behavior. Learning in a changing worldMost days, you drive home along a familiar route. But today, something unexpected happens. The city has opened a new street, offering the possibility of a shortcut. Another day, a new intersection sends you down a road you didn't intend. Often a traffic jam severely alters the time it takes you to reach your destination. Whether you are a human driving home, a monkey foraging for food, or a rat navigating a maze, unexpected changes in the world necessitate a shift in behavioral policy-rules that guide decisions based on prior knowledge-and potentially promote learning. Changes force agents to engage learning systems, switch mental states, and shift attention, among other adjustments, and recent work has examined their physiological substrates [1,2]. Yet the loci of change detection within the brain remain unidentified. Here we propose that the posterior cingulate cortex (CGp) plays a key role in altering behavior in response to unexpected change. It may be that this region, which consumes more energy than any other cortical area [3], must do so just to keep pace with a dynamically changing world.Corresponding author: Platt, M.L., platt@neuro.duke.edu. Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. NIH Public Access Author ManuscriptTrends Cogn Sci. Author manuscript; available in PMC 2012 April 1. Since the discovery that dopaminergic neurons respond to rewarding events by signaling the difference between expected and received rewards-the so-called "reward prediction error" (RPE)-theories of reinforcement learning (RL) have come to dominate discussions of learning and conditioning [4,5]. In typical RL algorithms, learning is incremental and only slowly converges on stable behavior [5]. In an environment rapidly alternating among several fixed but distinct reward contingencies, crude RL agents might find themselves forever playing catch-up, unable to do more than gradually adjust in response to abr...

SUMMARY Decision-makers are curious and consequently value advance information about future events. We made use of this fact to test competing theories of value representation in Area 13 of orbitofrontal cortex (OFC). In a new task, we found that monkeys reliably sacrificed primary reward (water) to view advance information about gamble outcomes. While monkeys integrated information value with primary reward value to make their decisions, OFC neurons had no systematic tendency to integrate these variables, instead encoding them in orthogonal manners. These results suggest that the predominant role of the OFC is to encode variables relevant for learning, attention, and decision-making rather than integrating them into a single scale of value. They also suggest that OFC may be placed at a relatively early stage in the hierarchy of information-seeking decisions, before evaluation is complete. Thus, our results delineate a circuit for information-seeking decisions and suggest a neural basis for curiosity.

The dorsal anterior cingulate cortex (dACC) has attracted great interest from neuroscientists because it is associated with so many important cognitive functions. Despite, or perhaps because of, its rich functional repertoire, we lack a single comprehensive view of its function. Most research has approached this puzzle from the top down, using aggregate measures such as neuroimaging. We provide a view from the bottom up, with a focus on singleunit responses and anatomy. We summarize the strengths and weaknesses of the three major approaches to characterizing the dACC: as a monitor, as a controller, and as an economic structure. We argue that neurons in the dACC are specialized for representing contexts, or task-state variables relevant for behavior, and strategies, or aspects of future plans. We propose that dACC neurons link contexts with strategies by integrating diverse taskrelevant information to create a rich representation of task space and exert high-level and abstract control over decision and action.

The core feature of an economic exchange is a decision to trade one good for another, based on a comparison of relative value. Economists have long recognized, however, that the value an individual ascribes to a good during decision making (i.e., their relative willingness to trade for that good) does not always map onto the reward they actually experience. Here, we show that experienced value and decision value are represented in distinct regions of ventromedial prefrontal cortex (VMPFC) during the passive consumption of rewards. Participants viewed two categories of rewards-images of faces that varied in their attractiveness and monetary gains and losses-while being scanned using functional magnetic resonance imaging. An independent market task, in which participants exchanged some of the money that they had earned for brief views of attractive faces, determined the relative decision value associated with each category. We found that activation of anterior VMPFC increased with increasing experienced value, but not decision value, for both reward categories. In contrast, activation of posterior VMPFC predicted each individual's relative decision value for face and monetary stimuli. These results indicate not only that experienced value and decision value are represented in distinct regions of VMPFC, but also that decision value signals are evident even in the absence of an overt choice task. We conclude that decisions are made by comparing neural representations of the value of different goods encoded in posterior VMPFC in a common, relative currency.