Neural Circuitry of Reward Prediction Error

Watabe-Uchida, Mitsuko; Eshel, Neir; Uchida, Naoshige

doi:10.1146/annurev-neuro-072116-031109

Cited by 315 publications

(317 citation statements)

References 126 publications

Supporting

Mentioning

280

Contrasting

Order By: Relevance

“…While our data demonstrate that the caloric content of food rapidly entrains the inhibition of AgRP neurons as a response to food cues, the function of this rapid sensory inhibition remains unknown. Since these neurons transmit a negative valence signal to drive food intake (Betley et al, 2015), it is possible that the rebound in AgRP neuron activity that occurs when a non-nutritive substance is identified serves as a reward prediction error (Schultz et al, 1997; Watabe-Uchida et al, 2017) that devalues non-food substances. This would be conceptually similar to the function of dopamine neurons in the ventral tegmental area, that first fire upon presentation of a reward, but over time fire instead at presentation of a cue predicting a reward.…”

Section: Discussionmentioning

confidence: 99%

Nutritive, Post-ingestive Signals Are the Primary Regulators of AgRP Neuron Activity

2017

View full text Add to dashboard Cite

SUMMARY The brain regulates food intake by processing sensory cues and peripheral physiological signals, but the neural basis of this integration remains unclear. Hypothalamic, agouti-related protein (AgRP)-expressing neurons are critical regulators of food intake. AgRP neuron activity is high during hunger, and is rapidly reduced by the sight and smell of food. Here, we revealed two distinct components of AgRP neuron activity regulation: a rapid, but transient sensory-driven signal; and a slower, sustained calorie-dependent signal. We discovered that nutrients are necessary and sufficient for sustained reductions in AgRP neuron activity, and that activity reductions are proportional to the calories obtained. This change in activity is recapitulated by exogenous administration of gut-derived satiation signals. Furthermore, we showed that the nutritive value of food trains sensory systems – in a single trial – to drive rapid, anticipatory AgRP neuron activity inhibition. Together, these data demonstrate that nutrients are the primary regulators of AgRP neuron activity.

show abstract

Section: Discussionmentioning

confidence: 99%

Nutritive, Post-ingestive Signals Are the Primary Regulators of AgRP Neuron Activity

2017

View full text Add to dashboard Cite

show abstract

“…For example, they could be provided by direct feedback from the aforementioned higher-order cortical areas, or they could be derived from sub-cortical areas that are implicated in attention and behavioral updating during learning (Wimmer et al, 2015). Modulatory reinforcement signals that are associated with behavioral outcome could also play a major role (Pi et al, 2013;Watabe-Uchida et al, 2017). Indeed, reward-related response modulation has been observed in S1 (Lacefield et al, 2019), and was found to promote cortical plasticity processes related to visual response tuning in primary visual cortex (Goltstein et al, 2013).…”

Section: Discussionmentioning

confidence: 99%

Dynamic perceptual feature selectivity in primary somatosensory cortex upon reversal learning

Chéreau¹,

Bawa²,

Fodoulian³

et al. 2019

Preprint

View full text Add to dashboard Cite

Neurons in primary sensory cortex encode a variety of stimulus features upon perceptual learning. However, it is unclear whether the acquired stimulus selectivity remains stable when the same input is perceived in a different context. Here, we monitored the activity of individual neurons in the mouse primary somatosensory cortex in a reward-based texture discrimination task. We tracked their stimulus selectivity before and after changing reward contingencies and found that many neurons are dynamically selective. A subpopulation of neurons regains selectivity contingent on stimulus-value. These value-sensitive neurons forecast the onset of learning by displaying a distinct and transient increase in activity, depending on past behavioral experience. Thus, stimulus encoding during perceptual learning is dynamic and largely relies on behavioral contingencies, even in primary sensory cortex.

show abstract

“…Besides these "phasic" responses, dopamine neurons typically maintain relatively low and stable baseline firing rates between these events. These response patterns have been observed in a number of animal species and in different task conditions (Bayer and Glimcher, 2005;Clark et al, 2012;Watabe-Uchida et al, 2017;Wenzel et al, 2015), and the RPE hypothesis has greatly impacted our understanding of dopamine functions in the brain. However, many of these experiments have employed relatively simple behavioral paradigms using discrete stimuli and outcomes.…”

Section: Introductionmentioning

confidence: 95%

A unified framework for dopamine signals across timescales

Kim

Malik

Mikhael

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Rapid phasic activity of midbrain dopamine neurons are thought to signal reward prediction errors (RPEs), resembling temporal difference errors used in machine learning. Recent studies describing slowly increasing dopamine signals have instead proposed that they represent state values and arise independently from somatic spiking activity. Here, we developed novel experimental paradigms using virtual reality that disambiguate RPEs from values. We examined the dopamine circuit activity at various stages including somatic spiking, axonal calcium signals, and striatal dopamine concentrations. Our results demonstrate that ramping dopamine signals are consistent with RPEs rather than value, and this ramping is observed at all the stages examined. We further show that ramping dopamine signals can be driven by a dynamic stimulus that indicates a gradual approach to a reward. We provide a unified computational understanding of rapid phasic and slowly ramping dopamine signals: dopamine neurons perform a derivative-like computation over values on a moment-by-moment basis.

show abstract

Neural Circuitry of Reward Prediction Error

Cited by 315 publications

References 126 publications

Nutritive, Post-ingestive Signals Are the Primary Regulators of AgRP Neuron Activity

Nutritive, Post-ingestive Signals Are the Primary Regulators of AgRP Neuron Activity

Dynamic perceptual feature selectivity in primary somatosensory cortex upon reversal learning

A unified framework for dopamine signals across timescales

Contact Info

Product

Resources

About