Orbitofrontal neurons infer the value and identity of predicted outcomes

Stalnaker, Thomas A.; Cooch, Nisha K.; McDannald, Michael A.; Liu, Tzu-Lan; Wied, Heather M.; Schoenbaum, Geoffrey

doi:10.1038/ncomms4926

Cited by 103 publications

(120 citation statements)

References 42 publications

Supporting

Mentioning

106

Contrasting

Order By: Relevance

“…At the level of single neurons, such a coding scheme could be implemented by units signaling both value and identity or an interaction between the two. Intriguingly, such neurons have recently been identified in the rat OFC (12), and it is likely that other features of the expected outcome, such as reward location, behavioral responses, and other valueless features, are also embedded in these complex predictive codes (3,(33)(34)(35). Thus, our findings indicate that predictive outcome representations in the OFC are much more complex than previously thought, and provide critical empirical support for recent proposals suggesting that the OFC plays a fundamental role in model-based behavior by tracking the contents and states of the environment and task structure (36,37).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Identity-specific coding of future rewards in the human orbitofrontal cortex

Howard

Gottfried

Tobler

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

192

180

View full text Add to dashboard Cite

Nervous systems must encode information about the identity of expected outcomes to make adaptive decisions. However, the neural mechanisms underlying identity-specific value signaling remain poorly understood. By manipulating the value and identity of appetizing food odors in a pattern-based imaging paradigm of human classical conditioning, we were able to identify dissociable predictive representations of identity-specific reward in orbitofrontal cortex (OFC) and identity-general reward in ventromedial prefrontal cortex (vmPFC). Reward-related functional coupling between OFC and olfactory (piriform) cortex and between vmPFC and amygdala revealed parallel pathways that support identityspecific and -general predictive signaling. The demonstration of identity-specific value representations in OFC highlights a role for this region in model-based behavior and reveals mechanisms by which appetitive behavior can go awry.reward value | associative learning | ventromedial prefrontal cortex | olfaction | multivoxel pattern analysis P redictive representations of future outcomes are critical for guiding adaptive behavior. To choose different types of rewards, such as food, shelter, and mates, it is essential that predictive signals contain specific information about the identity of those outcomes. Food rewards differ dramatically in their nutritional composition, and identity-specific cues allow differential foraging depending on current needs of the organism. The absence of precise mappings between predictive reward signals and their intended outcomes would have devastating effects on foodbased decisions.Despite the ecological relevance of outcome-specific predictive coding, which can be observed even in Drosophila (1), most research in human and nonhuman primates has focused on "common currency" signals of economic values in the orbitofrontal cortex (OFC) (2, 3) and ventromedial prefrontal cortex (vmPFC) (4-8). These signals, which by definition are independent of the specific nature of the reward, can be used to compare and choose between alternative outcomes, but are unable to inform expectations about the specific identity of the outcome. For this, identity-specific representations that conjointly represent information about both affective value (how good is it?) and outcome identity (what is it?) are necessary. Recent data suggest that the OFC is involved in signaling information about specific outcomes (9-14). For instance, many OFC neurons signal both the value and the identity of the predicted outcome (12), and OFC lesions diminish the effects of outcome identity (but not general affective value) on conditioned behavior (13).Recent imaging work has also begun to address how the human brain encodes predictive information about rewarding outcomes. One study (9) used a functional magnetic resonance imaging (fMRI) adaptation paradigm to provide evidence for identity-based codes for reward in the OFC. Another investigation (4) used fMRI data from a willingness to pay auction combined with decoding techniques to reveal...

show abstract

Section: Discussionmentioning

confidence: 99%

“…Recent data suggest that the OFC is involved in signaling information about specific outcomes (9)(10)(11)(12)(13)(14). For instance, many OFC neurons signal both the value and the identity of the predicted outcome (12), and OFC lesions diminish the effects of outcome identity (but not general affective value) on conditioned behavior (13).…”

mentioning

confidence: 99%

Identity-specific coding of future rewards in the human orbitofrontal cortex

Howard

Gottfried

Tobler

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

192

180

View full text Add to dashboard Cite

show abstract

“…In particular, high-level option representations are proposed to depend crucially on lateral prefrontal cortex (Botvinick et al, 2009), with input from orbitofrontal and perhaps medial prefrontal regions (Holroyd and Yeung, 2012), and with representations in these regions influencing action selection in the basal ganglia under the influence of reward. This network description has recently been extended to include the proposal that the orbitofrontal cortex (OFC) represents states that afford specific options (Stalnaker et al, 2014;Wilson et al, 2014). In this way, knowledge of the physiology of prediction, evaluation, and action selection from reinforcement learning research may deliver valuable insights into the mechanisms by which outcomes may globally influence action selection and cognitive control.…”

Section: Hrl and Cognitive Controlmentioning

confidence: 98%

“…Within our HRL framework, modulation of striatal representations of dlPFC input by OFC projections is particularly interesting in relation to the concepts of option availability and of action selection within options (Stalnaker et al, 2014;Wilson et al, 2014). The OFC has been associated with cod ing the present states, i.e., states affording different options (Wilson et al, 2014).…”

Section: Multiple Projection Hierarchies In the Basal Gangliamentioning

confidence: 99%

The role of prediction and outcomes in adaptive cognitive control

Schiffer

Waszak

Yeung

2015

Journal of Physiology-Paris

View full text Add to dashboard Cite

Humans adaptively perform actions to achieve their goals. This flexible behaviour requires two core abilities: the ability to anticipate the outcomes of candidate actions and the ability to select and implement actions in a goal-directed manner. The ability to predict outcomes has been extensively researched in reinforcement learning paradigms, but this work has often focused on simple actions that are not embedded in hierarchical and sequential structures that are characteristic of goal-directed human behaviour. On the other hand, the ability to select actions in accordance with high-level task goals, particularly in the presence of alternative responses and salient distractors, has been widely researched in cognitive control paradigms. Cognitive control research, however, has often paid less attention to the role of action outcomes. The present review attempts to bridge these accounts by proposing an outcome-guided mechanism for selection of extended actions. Our proposal builds on constructs from the hierarchical reinforcement learning literature, which emphasises the concept of reaching and evaluating informative states, i.e., states that constitute subgoals in complex actions. We develop an account of the neural mechanisms that allow outcome-guided action selection to be achieved in a network that relies on projections from cortical areas to the basal ganglia and back-projections from the basal ganglia to the cortex. These cortico-basal ganglia-thalamo-cortical 'loops' allow convergence - and thus integration - of information from non-adjacent cortical areas (for example between sensory and motor representations). This integration is essential in action sequences, for which achieving an anticipated sensory state signals the successful completion of an action. We further describe how projection pathways within the basal ganglia allow selection between representations, which may pertain to movements, actions, or extended action plans. The model lastly envisages a role for hierarchical projections from the striatum to dopaminergic midbrain areas that enable more rostral frontal areas to bias the selection of inputs from more posterior frontal areas via their respective representations in the basal ganglia.

show abstract

“…The orbitofrontal portion of the prefrontal cortex (OFC) in particular has been found to have a role in decisions based on outcome value and reward expectations (Kepecs et al, 2008;McDannald et al, 2014;Padoa-Schioppa and Cai, 2011;Stalnaker et al, 2014). When rewards are uncertain, the magnitude of response in OFC has been shown to modulate proportionally with risk (Tobler et al, 2009), and lesions of OFC disrupt decisions that involve risk (Hsu et al, 2005).…”

Section: Introductionmentioning

confidence: 99%

Consequences of Adolescent Ethanol Consumption on Risk Preference and Orbitofrontal Cortex Encoding of Reward

McMurray

Amodeo

Roitman

2015

Neuropsychopharmacol

View full text Add to dashboard Cite

Critical development of the prefrontal cortex occurs during adolescence, a period of increased independence marked by decision making that often includes engagement in risky behaviors, such as substance use. Consumption of alcohol during adolescence has been associated with increased impulsivity that persists across the lifespan, an effect which may be caused by long-term disruptions in cortical processing of rewards. To determine if alcohol consumption alters cortical encoding of rewards of different sizes and probabilities, we gave rats limited access to alcohol in gelatin during adolescence only. In adulthood, we recorded the electrophysiological activity of individual neurons of the orbitofrontal cortex while rats performed a risk task that varied the level of risk from day-to-day. Rats that had consumed higher levels of alcohol showed increased risk preference in the task compared with control and low alcohol-consuming rats. Patterns of neuronal responses were identified using principal component analysis. Of the multiple patterns observed, only one was modulated by adolescent alcohol consumption and showed strongest modulation after reward receipt. This subpopulation of neurons showed blunted firing rates following rewards in alcohol-consuming rats, suggesting a mechanism through which adolescent alcohol exposure may have lasting effects on reward processing in the context of decision making. The differences in OFC responses between high alcohol consumers and control animals not given access to alcohol support the idea that, regardless of potential variability in innate alcohol preferences, voluntary consumption of alcohol during adolescence biases choice patterns longitudinally through alterations in cortical function.

show abstract

Orbitofrontal neurons infer the value and identity of predicted outcomes

Cited by 103 publications

References 42 publications

Identity-specific coding of future rewards in the human orbitofrontal cortex

Identity-specific coding of future rewards in the human orbitofrontal cortex

The role of prediction and outcomes in adaptive cognitive control

Consequences of Adolescent Ethanol Consumption on Risk Preference and Orbitofrontal Cortex Encoding of Reward

Contact Info

Product

Resources

About