Mesolimbic dopamine release conveys causal associations

Jeong, Huijeong; Taylor, Annie; Floeder, Joseph R; Lohmann, Martin; Mihalaş, Ştefan; Wu, Brenda; Zhou, Mingkang; Burke, Dennis A; Namboodiri, Vijay Mohan K.

doi:10.1126/science.abq6740

Cited by 109 publications

(211 citation statements)

References 96 publications

Supporting

Mentioning

133

Contrasting

Order By: Relevance

“…Indeed, DA encodes causal associations regardless of the unconditioned stimulus (i.e. rewarding or aversive), at least in regions such as the nucleus accumbens core (Jeong et al, 2022) but also systemically (Roughley et al, 2021). But how information from these DA neuron types converges to assign and continuously update reward and/or fear 'value' is unknown.…”

Section: Discussionmentioning

confidence: 99%

50 Years of Research on Dopamine’s Role in Passive Aversive Conditioning and Extinction: A Systematic Review

Hamati

Ahrens

Holahan

et al. 2023

Preprint

View full text Add to dashboard Cite

Dopamine, a catecholamine neurotransmitter, has historically been associated with the encoding of reward, whereas its role in aversion has received less attention. Here, we systematically gather the vast evidence for a role of dopamine in the simplest form of aversive learning: passive aversive conditioning and extinction. In the past, crude methods were used to augment or inhibit dopamine in order to study its relationship with fear conditioning and extinction. More advanced techniques such as conditional genetic, chemogenic, and optogenetic approaches now provide causal evidence for dopamine’s role in these learning processes. Dopamine neurons encode fear and extinction-related information and convey the signal via activation of D1, D2, D3, and D4 receptor sites particularly in the amygdala, prefrontal cortex, and striatum in order to continuously form, consolidate, retrieve, and update fear and extinction memory in a dynamic and reciprocal manner. Based on the reviewed literature, we conclude that dopamine is crucial for the encoding of passive aversive conditioning and contributes in a way that is comparable to its role in encoding reward.

show abstract

Section: Discussionmentioning

confidence: 99%

50 Years of Research on Dopamine’s Role in Passive Aversive Conditioning and Extinction: A Systematic Review

Hamati

Ahrens

Holahan

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…While phasic dopamine responses have long been assumed to encode a scalar prediction error value as calculated by a single temporal difference learning agent (Montague et al, 1996;Schultz et al, 1997), the growing wealth of studies showing the existence of multiple simultaneous prediction error call for a reinterpretation of the role of dopamine in signaling prediction errors (Langdon et al, 2018). In line with this proposal, recently published findings suggest that dopamine function might actually be better understood as retrospective causal inference (i.e., credit assignment), contesting the prospective temporal difference reinforcement learning framework (Jeong et al, 2022). While this work calls the prevailing reward prediction error hypothesis of mesolimbic dopamine into question, its proposed model is highly consistent with and reliant on the notion of surprise to solve the (temporal) credit assignment problem.…”

Section: Studies Investigating the Distinction Between Model-free And...mentioning

confidence: 96%

Surprise-minimization as a solution to the structural credit assignment problem

Wurm¹,

Ernst²,

Steinhauser³

2023

Preprint

View full text Add to dashboard Cite

The structural credit assignment problem arises when the causal structure between actions and subsequent outcomes is hidden from direct observation. To solve this problem and enable goal-directed behavior, an agent has to infer structure and form a representation thereof. In the scope of this study, we investigate a possible solution in the human brain. We recorded behavioral and electrophysiological data from human participants in a novel variant of the bandit task, where multiple actions lead to multiple outcomes. Crucially, the mapping between actions and outcomes was hidden and not instructed to the participants. Human choice behavior revealed clear hallmarks of credit assignment and learning. Moreover, a computational model which formalizes action selection as the competition between multiple representations of the hidden structure was fit to account for participants data. Starting in a state of uncertainty about the correct representation, the central mechanism of this model is the arbitration of action control towards the representation which minimizes surprise about outcomes. Crucially, single-trial latent-variable analysis reveals that the neural patterns clearly support central quantitative predictions of this surprise minimization model. The results suggest that posterior activity is not only related to reinforcement learning under correct as well as incorrect task representations but also reflects central mechanisms of credit assignment and representation learning.

show abstract

“…Historically, dopamine prediction errors have been thought to contribute to cue-reward learning by endowing cues with a model-free, scalar value [29][30][31][32][33][34] . However, recent studies have revealed that these phasic signals can also support the development of model-based associations [36][37][38][39][40][41][42][43][44][45][46][47] . For example, VTADA neurons are necessary for the development of associations between cues and sensory-specific representations of rewards 38,40 .…”

Section: Confirming the Presence Of A Novel Dopaminergic Projection F...mentioning

confidence: 99%

“…Further, the wider circuity supporting reward learning in the LH is unclear. One candidate for this is input from dopamine neurons in the ventral tegmental area (VTA), which we now understand to be capable of supporting both model-free and model-based associative learning [29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47] . However, there are very few studies that have suggested the presence of a projection from dopamine neurons to VTA 48,49 , and none that explore the function of this projection.…”

Section: Introductionmentioning

confidence: 99%

A novel hypothalamic-midbrain circuit for model-based learning

Hoang

Munier

Verghese

et al. 2023

Preprint

View full text Add to dashboard Cite

Behavior is often dichotomized into model-free and model-based systems1, 2. Model-free behavior prioritizes associations that have high value, regardless of the specific consequence or circumstance. In contrast, model-based behavior involves considering all possible outcomes to produce behavior that best fits the current circumstance. We typically exhibit a mixture of these behaviors so we can trade-off efficiency and flexibility. However, substance use disorder shifts behavior more strongly towards model-free systems, which produces a difficulty abstaining from drug-seeking due to an inability to withhold making the model-free high-value response3–10. The lateral hypothalamus (LH) is implicated in substance use disorder11–17and we have demonstrated that this region is critical to Pavlovian cue-reward learning18, 19. However, it is unknown whether learning occurring in LH is model-free or model-based, where the necessary teaching signal comes from to facilitate learning in LH, and whether this is relevant for learning deficits that drive substance use disorder. Here, we reveal that learning occurring in the LH is model-based. Further, we confirm the existence of an understudied projection extending from dopamine neurons in the ventral tegmental area (VTA) to the LH and demonstrate that this input underlies model-based learning in LH. Finally, we examine the impact of methamphetamine self-administration on LH-dependent model-based processes. These experiments reveal that a history of methamphetamine administration enhances the model-based control that Pavlovian cues have over decision-making, which was accompanied by a bidirectional strengthening of the LH to VTA circuit. Together, this work reveals a novel bidirectional circuit that underlies model-based learning and is relevant to the behavioral and cognitive changes that arise with substance use disorders. This circuit represents a new addition to models of addiction, which focus on instrumental components of drug addiction and increases in model-free habits after drug exposure3–10.

show abstract

Mesolimbic dopamine release conveys causal associations

Cited by 109 publications

References 96 publications

50 Years of Research on Dopamine’s Role in Passive Aversive Conditioning and Extinction: A Systematic Review

50 Years of Research on Dopamine’s Role in Passive Aversive Conditioning and Extinction: A Systematic Review

Surprise-minimization as a solution to the structural credit assignment problem

A novel hypothalamic-midbrain circuit for model-based learning

Contact Info

Product

Resources

About