Dynamic Gain Control of Dopamine Delivery in Freely Moving Animals

Montague, P. Read; McClure, Samuel M.; Baldwin, Philip R.; Phillips, Paul E. M.; Budygin, Evgeny A.; Stuber, Garret D.; Kilpatrick, Michaux; Wightman, R. Mark

doi:10.1523/jneurosci.4279-03.2004

Cited by 157 publications

(171 citation statements)

References 34 publications

Supporting

Mentioning

151

Contrasting

Unclassified

Order By: Relevance

“…Stimulation-evoked dopamine release declines because of a restricted releasable pool of dopamine (20,29,30). A mathematical model proposed by Montague and coworkers predicts diminished dopamine release over the long term of the 200 ICSS trials that is quite similar to our experimental results.…”

Section: Discussionsupporting

confidence: 88%

See 1 more Smart Citation

Dynamic changes in accumbens dopamine correlate with learning during intracranial self-stimulation

Owesson-White

Cheer²,

Beyene³

et al. 2008

Proc. Natl. Acad. Sci. U.S.A.

122

167

View full text Add to dashboard Cite

Dopamine in the nucleus accumbens (NAc) is an important neurotransmitter for reward-seeking behaviors such as intracranial selfstimulation (ICSS), although its precise role remains unclear. Here, dynamic fluctuations in extracellular dopamine were measured during ICSS in the rat NAc shell with fast-scan cyclic voltammetry at carbon-fiber microelectrodes. Rats were trained to press a lever to deliver electrical stimulation to the substantia nigra (SNc)/ ventral tegmental area (VTA) after the random onset of a cue that predicted reward availability. Latency to respond after cue onset significantly declined across trials, indicative of learning. Dopamine release was evoked by the stimulation but also developed across trials in a time-locked fashion to the cue. Once established, the cue-evoked dopamine transients continued to grow in amplitude, although they were variable from trial to trial. The emergence of cue-evoked dopamine correlated with a decline in electrically evoked dopamine release. Extinction of ICSS resulted in a significant decline in goal-directed behavior coupled to a significant decrease in cue-evoked phasic dopamine across trials. carbon-fiber electrode ͉ cyclic voltammetry ͉ extinction ͉ nucleus accumbens shell ͉ reward I ntracranial self-stimulation (ICSS) was discovered in 1954 (1). In this paradigm, a rat depresses a lever to deliver an electric shock to electrodes implanted within the brain. Extensive mapping studies by Olds and Olds later showed that the neuroanatomical region supporting ICSS centered in the posterior MFB region of the lateral hypothalamus (2). This finding provoked considerable interest, because it identified a brain reward pathway that could be centrally activated without the need for sensory stimulation (3, 4). Although a role for several neurotransmitters has been implicated in ICSS, dopamine appears to play a primary role (5, 6), leading to the view that dopaminergic signaling is essential during goal-directed behaviors. Indeed, it was postulated that increased dopaminergic neurotransmission was necessary for the reinforcement of reward-related behavior (7).More recently, electrophysiological studies in primates have provided new insight into the role of dopaminergic neurons in reward processing (8). In response to unexpected rewards, dopamine neurons exhibit phasic firing. However, when an animal learns that a cue predicts reward, the burst of neuronal firing switches to the onset of the cue (9-12). Responses to the cue increase with repeated trials, and these paired responses of midbrain dopamine neurons follow the expectations of models of associative learning in which dopamine signaling is a reward-prediction error (12,13). Similar responses to conditioned stimuli that predict reward have also been observed for midbrain dopaminergic neurons in rats (14).A phasic increase in dopamine neuronal firing should lead to a dopamine concentration transient in terminal areas such as the nucleus accumbens (NAc). Indeed, using fast-scan cyclic voltammetry at carbon-fiber microe...

show abstract

Section: Discussionsupporting

confidence: 88%

“…Superimposed on the experimental data are a line computed with a neurochemical model that predicts dopamine release with repeated stimulation (see Fig. 3D and Discussion) (20).…”

Section: Resultsmentioning

confidence: 97%

Dynamic changes in accumbens dopamine correlate with learning during intracranial self-stimulation

Owesson-White

Cheer²,

Beyene³

et al. 2008

Proc. Natl. Acad. Sci. U.S.A.

122

167

View full text Add to dashboard Cite

show abstract

“…To better understand the kinetics of DA delivery in the basal ganglia in response to stimulating pulses applied to the VTA/SNc, Montague et al (2004b) created the ''kick and relax'' model. The authors used the Michaelis-Menten first order differential equation which is often used in biochemistry to describe enzymatic reactions.…”

Section: Model Of Dopamine Releasementioning

confidence: 99%

“…In this study, the extracellular DA dynamics due to DA diffusion as well as DA degradation were calculated. The former was calculated based on the Michaelis-Menten equation and the latter was based on diffusion equations, similar to the 'kick and relax' model of Montague et al (2004b) presented in the previous section. The change in DA concentration was then introduced multiplicatively into the STDP learning rule to simulate experimental data.…”

Section: Models Of the Effects Of Dopamine Releasementioning

confidence: 99%

Computational models of reinforcement learning: the role of dopamine as a reward signal

2010

View full text Add to dashboard Cite

Reinforcement learning is ubiquitous. Unlike other forms of learning, it involves the processing of fast yet content-poor feedback information to correct assumptions about the nature of a task or of a set of stimuli. This feedback information is often delivered as generic rewards or punishments, and has little to do with the stimulus features to be learned. How can such low-content feedback lead to such an efficient learning paradigm? Through a review of existing neuro-computational models of reinforcement learning, we suggest that the efficiency of this type of learning resides in the dynamic and synergistic cooperation of brain systems that use different levels of computations. The implementation of reward signals at the synaptic, cellular, network and system levels give the organism the necessary robustness, adaptability and processing speed required for evolutionary and behavioral success.

show abstract

“…In support of this claim, direct recordings of spike activity in mesencephalic dopaminergic neurons in nonhuman primates demonstrate that these neurons encode prediction errors in future reward delivery (8-10, 13, 14) and they may also encode other computations relevant for reward-guided actions (1,(15)(16)(17). However, action potential production in brainstem dopaminergic neurons can only be part of the story because activity in parent axons must be converted to changes in neurotransmitter release at synaptic terminals to have any impact on downstream neural systems (1,18). There have been no direct measurements of dopamine release in human striatum that tests these ideas directly.…”

mentioning

confidence: 99%

Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward

Kishida

Sáez

Lohrenz

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

184

283

View full text Add to dashboard Cite

In the mammalian brain, dopamine is a critical neuromodulator whose actions underlie learning, decision-making, and behavioral control. Degeneration of dopamine neurons causes Parkinson's disease, whereas dysregulation of dopamine signaling is believed to contribute to psychiatric conditions such as schizophrenia, addiction, and depression. Experiments in animal models suggest the hypothesis that dopamine release in human striatum encodes reward prediction errors (RPEs) (the difference between actual and expected outcomes) during ongoing decision-making. Blood oxygen level-dependent (BOLD) imaging experiments in humans support the idea that RPEs are tracked in the striatum; however, BOLD measurements cannot be used to infer the action of any one specific neurotransmitter. We monitored dopamine levels with subsecond temporal resolution in humans (n = 17) with Parkinson's disease while they executed a sequential decision-making task. Participants placed bets and experienced monetary gains or losses. Dopamine fluctuations in the striatum fail to encode RPEs, as anticipated by a large body of work in model organisms. Instead, subsecond dopamine fluctuations encode an integration of RPEs with counterfactual prediction errors, the latter defined by how much better or worse the experienced outcome could have been. How dopamine fluctuations combine the actual and counterfactual is unknown. One possibility is that this process is the normal behavior of reward processing dopamine neurons, which previously had not been tested by experiments in animal models. Alternatively, this superposition of error terms may result from an additional yet-to-be-identified subclass of dopamine neurons.dopamine | reward prediction error | counterfactual prediction error | decision-making | human fast-scan cyclic voltammetry

show abstract

Dynamic Gain Control of Dopamine Delivery in Freely Moving Animals

Cited by 157 publications

References 34 publications

Dynamic changes in accumbens dopamine correlate with learning during intracranial self-stimulation

Dynamic changes in accumbens dopamine correlate with learning during intracranial self-stimulation

Computational models of reinforcement learning: the role of dopamine as a reward signal

Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward

Contact Info

Product

Resources

About