Orbitofrontal Cortex Supports Behavior and Learning Using Inferred But Not Cached Values

Jones, Joe M.; Esber, Guillem R.; McDannald, Michael A.; Gruber, Aaron J.; Hernández, Alex; Mirenzi, Aaron; Schoenbaum, Geoffrey

doi:10.1126/science.1227489

Cited by 297 publications

(320 citation statements)

References 31 publications

Supporting

Mentioning

284

Contrasting

Unclassified

Order By: Relevance

“…Clearly, the rapid approach after the motivational shift cannot be accounted for by a cached stimulus value -this would require multiple iterations of sampling the salt water in the salt-hungry state before the new positive prediction errors could update the stimulus value sufficiently to make it attractive. Instead, this experiment suggests that the animals learned the identity of the outcome associated with the stimulus, and in the novel salt-hungry state were able to use this to infer the new value of the stimulus given the new value of the outcome it predicted (Schoenbaum et al, 2009;Jones et al, 2012;Dayan and Berridge, 2013). …”

Section: Motivational Shiftsmentioning

confidence: 99%

“…Then, an identity shift occurs: the compound predicts the new reward, which was measured to be of equal value. Thus, there is no value prediction error (equation 7), yet animals are sensitive to such shifts (Seligman, 1970;Bouton, 2006;McDannald et al, 2011;Takahashi et al, 2011;Jones et al, 2012), showing that they do represent and learn more features about the outcome than the scalar measure of how rewarding it is. Thus, while transreinforcer blocking (and value blocking more generally) supports model-free processes, identity unblocking is evidence for model-based processes in Pavlovian conditioning (McDannald et al, 2011).…”

Section: Outcome Identitymentioning

confidence: 99%

See 1 more Smart Citation

The role of learning-related dopamine signals in addiction vulnerability

Huys

Tobler

Hasler

et al. 2014

Progress in Brain Research

View full text Add to dashboard Cite

Psychostimulants such as methylphenidate (MPH) and antidepressants such as fluoxetine (FLX) are widely used in the treatment of various mental disorders or as cognitive enhancers. These medications are often combined, for example, to treat comorbid disorders. There is a considerable body of evidence from animal models indicating that individually these psychotropic medications can have detrimental effects on the brain and behavior, especially when given during sensitive periods of brain development. However, almost no studies investigate possible interactions between these drugs. This is surprising given that their combined neurochemical effects (enhanced dopamine and serotonin neurotransmission) mimic some effects of illicit drugs such as cocaine and amphetamine. Here, we summarize recent studies in juvenile rats on the molecular effects in the mid-and forebrain and associated behavioral changes, after such combination treatments. Our findings indicate that these combined MPH + FLX treatments can produce similar molecular changes as seen after cocaine exposure while inducing behavioral changes indicative of dysregulated mood and motivation, effects that often endure into adulthood. Tables: 2 References: 254 ABSTRACT Dopaminergic signals play a mathematically precise role in reward-related learning, and variations in dopaminergic signalling have been implicated in vulnerability to addiction. Here, we provide a detailed overview of the relationship between theoretical, mathematical and experimental accounts of phasic dopamine signalling, with implications for the role of learning-related dopamine signalling in addiction and related disorders. We describe the theoretical and behavioural characteristics of model-free learning based on errors in the prediction of reward, including step-by-step explanations of the underlying equations. We then use recent insights from an animal model that highlights individual variation in learning during a Pavlovian conditioning paradigm to describe overlapping aspects of incentive salience attribution and model-free learning. We argue that this provides a computationally coherent account of some features of addiction. CONTENTS

show abstract

Section: Motivational Shiftsmentioning

confidence: 99%

Section: Outcome Identitymentioning

confidence: 99%

The role of learning-related dopamine signals in addiction vulnerability

Huys

Tobler

Hasler

et al. 2014

Progress in Brain Research

View full text Add to dashboard Cite

show abstract

“…The literature suggests that the PFC can flexibly adjust outcome values in response to motivational or environmental changes even before experiencing the outcome (Corbit and Balleine 2003;Izquierdo et al 2004;Ostlund and Balleine 2005;Jones et al 2012;Takahashi et al 2013). Due to this characteristic of updating values independent of reward prediction errors, little attention has been paid to the possibility that the PFC may directly influence computations of DAergic error signals.…”

Section: Introductionmentioning

confidence: 99%

Prefrontal Regulation of Neuronal Activity in the Ventral Tegmental Area

Mizumori

2015

Cereb. Cortex

View full text Add to dashboard Cite

Growing evidence indicates that midbrain dopamine (DA) cells integrate reward expectancy-related information from the prefrontal cortex to properly compute errors in reward prediction. Here we investigated how 2 major prefrontal subregions, the orbitofrontal cortex (OFC) and the medial prefrontal cortex (mPFC), contributed to DAergic prediction errors while rats performed a delay discounting task on a T-maze. Most putative DA cells in the task showed phasic responses to salient cues that predicted delayed rewards, but not to the actual rewards. After temporary inactivation of the OFC, putative DA cells exhibited strikingly reduced phasic responses to reward-predicting cues but increased responses to rewards. In contrast, mPFC inactivation significantly elevated DA responses to both predictive cues and rewards. In addition, OFC, but not mPFC, inactivation disrupted the activity of putative non-DA cells that encoded expected reward values during waiting periods. These results suggest that the 2 prefrontal subregions differentially regulate DAergic prediction errors and the OFC conveys value signals to midbrain dopamine systems.

show abstract

“…Experiments in human and animal subjects have identified candidate brain areas for performing these different computations [1][2][3][4][5], including the orbitofrontal cortex (OFC) and ventral striatum (vStr) [6][7][8][9][10][11][12][13][14][15].…”

Section: Introductionmentioning

confidence: 99%

A functional difference in information processing between orbitofrontal cortex and ventral striatum during decision-making behaviour

Stott¹,

Redish

2014

Phil. Trans. R. Soc. B

View full text Add to dashboard Cite

Both orbitofrontal cortex (OFC) and ventral striatum (vStr) have been identified as key structures that represent information about value in decision-making tasks. However, the dynamics of how this information is processed are not yet understood. We recorded ensembles of cells from OFC and vStr in rats engaged in the spatial adjusting delay-discounting task , a decision-making task that involves a trade-off between delay to and magnitude of reward. Ventral striatal neural activity signalled information about reward before the rat's decision, whereas such reward-related signals were absent in OFC until after the animal had committed to its decision. These data support models in which vStr is directly involved in action selection, but OFC processes decision-related information afterwards that can be used to compare the predicted and actual consequences of behaviour.

show abstract

Orbitofrontal Cortex Supports Behavior and Learning Using Inferred But Not Cached Values

Cited by 297 publications

References 31 publications

The role of learning-related dopamine signals in addiction vulnerability

The role of learning-related dopamine signals in addiction vulnerability

Prefrontal Regulation of Neuronal Activity in the Ventral Tegmental Area

A functional difference in information processing between orbitofrontal cortex and ventral striatum during decision-making behaviour

Contact Info

Product

Resources

About