Dopamine errors drive excitatory and inhibitory components of backward conditioning in an outcome-specific manner

Seitz, Benjamin M; Hoang, Ivy B; DiFazio, Lauren E; Blaisdell, Aaron P.; Sharpe, Melissa J

doi:10.1016/j.cub.2022.06.035

Cited by 16 publications

(12 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our explanation is also consistent with dopamine ramps in the striatum reflecting a causal association between an action and reward (59). Finally, dopamine responses guide learning in a way that sometimes violates the predictions of model-free TDRL (17,(60)(61)(62)(63). Our proposal that the dopaminergic system conveys whether cues are meaningful causal targets, thereby promoting the learning of their causes, explains these results (fig.…”

Section: Discussionsupporting

confidence: 83%

Mesolimbic dopamine release conveys causal associations

Jeong

Taylor

Floeder

et al. 2022

Science

109

133

View full text Add to dashboard Cite

Learning to predict rewards based on environmental cues is essential for survival. It is believed that animals learn to predict rewards by updating predictions whenever the outcome deviates from expectations, and that such reward prediction errors (RPEs) are signaled by the mesolimbic dopamine system—a key controller of learning. However, instead of learning prospective predictions from RPEs, animals can infer predictions by learning the retrospective cause of rewards. Hence, whether mesolimbic dopamine instead conveys a causal associative signal that sometimes resembles RPE remains unknown. We developed an algorithm for retrospective causal learning and found that mesolimbic dopamine release conveys causal associations but not RPE, thereby challenging the dominant theory of reward learning. Our results reshape the conceptual and biological framework for associative learning.

show abstract

Section: Discussionsupporting

confidence: 83%

Mesolimbic dopamine release conveys causal associations

Jeong

Taylor

Floeder

et al. 2022

Science

109

133

View full text Add to dashboard Cite

show abstract

“…Historically, dopamine prediction errors have been thought to contribute to cue-reward learning by endowing cues with a model-free, scalar value [29][30][31][32][33][34] . However, recent studies have revealed that these phasic signals can also support the development of model-based associations [36][37][38][39][40][41][42][43][44][45][46][47] . For example, VTADA neurons are necessary for the development of associations between cues and sensory-specific representations of rewards 38,40 .…”

Section: Confirming the Presence Of A Novel Dopaminergic Projection F...mentioning

confidence: 99%

“…However, recent studies have revealed that these phasic signals can also support the development of model-based associations [36][37][38][39][40][41][42][43][44][45][46][47] . For example, VTADA neurons are necessary for the development of associations between cues and sensory-specific representations of rewards 38,40 . While there is a body of literature showing dopamine activity regulates LH function, this is largely based on local pharmacological manipulations through dopamine-receptor agonists and antagonists and focused on the canonical role of LH in regulating feeding 48,[62][63][64][65][66] .…”

Section: Confirming the Presence Of A Novel Dopaminergic Projection F...mentioning

confidence: 99%

“…Further, the wider circuity supporting reward learning in the LH is unclear. One candidate for this is input from dopamine neurons in the ventral tegmental area (VTA), which we now understand to be capable of supporting both model-free and model-based associative learning [29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47] . However, there are very few studies that have suggested the presence of a projection from dopamine neurons to VTA 48,49 , and none that explore the function of this projection.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A novel hypothalamic-midbrain circuit for model-based learning

Hoang

Munier

Verghese

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Behavior is often dichotomized into model-free and model-based systems1, 2. Model-free behavior prioritizes associations that have high value, regardless of the specific consequence or circumstance. In contrast, model-based behavior involves considering all possible outcomes to produce behavior that best fits the current circumstance. We typically exhibit a mixture of these behaviors so we can trade-off efficiency and flexibility. However, substance use disorder shifts behavior more strongly towards model-free systems, which produces a difficulty abstaining from drug-seeking due to an inability to withhold making the model-free high-value response3–10. The lateral hypothalamus (LH) is implicated in substance use disorder11–17and we have demonstrated that this region is critical to Pavlovian cue-reward learning18, 19. However, it is unknown whether learning occurring in LH is model-free or model-based, where the necessary teaching signal comes from to facilitate learning in LH, and whether this is relevant for learning deficits that drive substance use disorder. Here, we reveal that learning occurring in the LH is model-based. Further, we confirm the existence of an understudied projection extending from dopamine neurons in the ventral tegmental area (VTA) to the LH and demonstrate that this input underlies model-based learning in LH. Finally, we examine the impact of methamphetamine self-administration on LH-dependent model-based processes. These experiments reveal that a history of methamphetamine administration enhances the model-based control that Pavlovian cues have over decision-making, which was accompanied by a bidirectional strengthening of the LH to VTA circuit. Together, this work reveals a novel bidirectional circuit that underlies model-based learning and is relevant to the behavioral and cognitive changes that arise with substance use disorders. This circuit represents a new addition to models of addiction, which focus on instrumental components of drug addiction and increases in model-free habits after drug exposure3–10.

show abstract

“…Such stimulus-outcome memories are fundamental components of the internal model of environmental relationships, aka cognitive map 10 , we use to generate the predictions and inferences needed for many forms of flexible, advantageous decision making 8,9,11,12 . Although there is little known of how we form stimulus-outcome memories, recent evidence suggests dopamine might actually contribute [13][14][15][16][17][18][19][20][21][22][23][24] . New data have challenged the value-centric dogma of dopamine function, indicating it plays a much broader role in learning than originally thought [25][26][27][28][29] .…”

mentioning

confidence: 99%

Dopamine projections to the basolateral amygdala drive the encoding of identity-specific reward memories

Sias¹,

Jafar²,

Goodpaster³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

To make adaptive decisions, we build an internal model of associative relationships in the environment and use it to predict specific forthcoming outcomes. Detailed stimulus-outcome memories are a core feature of such cognitive maps, yet little is known of the neuronal systems that support their encoding. We used fiber photometry, cell-type and pathway-specific optogenetic manipulation, Pavlovian cue-reward conditioning, and a decision-making test in male and female rats, to reveal that ventral tegmental area dopamine (VTADA) projections to the basolateral amygdala (BLA) drive the encoding of stimulus-outcome memories. Dopamine is released in the BLA during stimulus-outcome pairing and VTADA→BLA activity is necessary and sufficient to link the identifying features of a reward to a predictive cue, but does not mediate general value or reinforcement. These data reveal a dopaminergic pathway for the learning that supports adaptive decision making and help understand how VTADA neurons achieve their emerging multifaceted role in learning.

show abstract

Dopamine errors drive excitatory and inhibitory components of backward conditioning in an outcome-specific manner

Cited by 16 publications

References 59 publications

Mesolimbic dopamine release conveys causal associations

Mesolimbic dopamine release conveys causal associations

A novel hypothalamic-midbrain circuit for model-based learning

Dopamine projections to the basolateral amygdala drive the encoding of identity-specific reward memories

Contact Info

Product

Resources

About