In attentional models of learning, associations between actions and subsequent rewards are stronger when outcomes are surprising, regardless of their valence. Despite the behavioral evidence that surprising outcomes drive learning, neural correlates of unsigned reward prediction errors remain elusive. Here we show that in a probabilistic choice task, trial-to-trial variations in preference track outcome surprisingness. Concordant with this behavioral pattern, responses of neurons in macaque (Macaca mulatta) dorsal anterior cingulate cortex (dACC) to both large and small rewards were enhanced when the outcome was surprising. Moreover, when, on some trials, probabilities were hidden, neuronal responses to rewards were reduced, consistent with the idea that the absence of clear expectations diminishes surprise. These patterns are inconsistent with the idea that dACC neurons track signed errors in reward prediction, as dopamine neurons do. Our results also indicate that dACC neurons do not signal conflict. In the context of other studies of dACC function, these results suggest a link between reward-related modulations in dACC activity and attention and motor control processes involved in behavioral adjustment. More speculatively, these data point to a harmonious integration between reward and learning accounts of ACC function on one hand, and attention and cognitive control accounts on the other.

When has the world changed enough to warrant a new approach? The answer depends upon current needs, behavioral flexibility, and prior knowledge about the environment. Formal approaches solve the problem by integrating the recent history of rewards, errors, uncertainty, and context via Bayesian inference to detect changes in the world and alter behavioral policy. Neuronal activity in posterior cingulate cortex (CGp)-a key node in the default network-is known to vary with learning, memory, reward, and task engagement. We propose that these modulations reflect the underlying process of change detection and motivate subsequent shifts in behavior. Learning in a changing worldMost days, you drive home along a familiar route. But today, something unexpected happens. The city has opened a new street, offering the possibility of a shortcut. Another day, a new intersection sends you down a road you didn't intend. Often a traffic jam severely alters the time it takes you to reach your destination. Whether you are a human driving home, a monkey foraging for food, or a rat navigating a maze, unexpected changes in the world necessitate a shift in behavioral policy-rules that guide decisions based on prior knowledge-and potentially promote learning. Changes force agents to engage learning systems, switch mental states, and shift attention, among other adjustments, and recent work has examined their physiological substrates [1,2]. Yet the loci of change detection within the brain remain unidentified. Here we propose that the posterior cingulate cortex (CGp) plays a key role in altering behavior in response to unexpected change. It may be that this region, which consumes more energy than any other cortical area [3], must do so just to keep pace with a dynamically changing world.Corresponding author: Platt, M.L., platt@neuro.duke.edu. Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. NIH Public Access Author ManuscriptTrends Cogn Sci. Author manuscript; available in PMC 2012 April 1. Since the discovery that dopaminergic neurons respond to rewarding events by signaling the difference between expected and received rewards-the so-called "reward prediction error" (RPE)-theories of reinforcement learning (RL) have come to dominate discussions of learning and conditioning [4,5]. In typical RL algorithms, learning is incremental and only slowly converges on stable behavior [5]. In an environment rapidly alternating among several fixed but distinct reward contingencies, crude RL agents might find themselves forever playing catch-up, unable to do more than gradually adjust in response to abr...

The dorsal anterior cingulate cortex (dACC) has attracted great interest from neuroscientists because it is associated with so many important cognitive functions. Despite, or perhaps because of, its rich functional repertoire, we lack a single comprehensive view of its function. Most research has approached this puzzle from the top down, using aggregate measures such as neuroimaging. We provide a view from the bottom up, with a focus on singleunit responses and anatomy. We summarize the strengths and weaknesses of the three major approaches to characterizing the dACC: as a monitor, as a controller, and as an economic structure. We argue that neurons in the dACC are specialized for representing contexts, or task-state variables relevant for behavior, and strategies, or aspects of future plans. We propose that dACC neurons link contexts with strategies by integrating diverse taskrelevant information to create a rich representation of task space and exert high-level and abstract control over decision and action.