Reinforcement determines the timing dependence of corticostriatal synaptic plasticity in vivo

Fisher, Simon D.; Robertson, Paul B.; Black, Melony J.; Redgrave, Peter; Sagar, Mark; Abraham, Wickliffe C.; Reynolds, John N. J.

doi:10.1038/s41467-017-00394-x

Cited by 88 publications

(92 citation statements)

References 58 publications

(112 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In slice preparations, NMDAR, Ca 2+ /calmodulin–dependent protein kinase II (CaMKII) and D1R regulate the enlargement of the dendritic spine, a structural basis for long-term potentiation of the D1R-expressing spiny projection neurons (D1-SPNs) in the NAc (Yagishita et al, 2014). Of note, the phasic burst of dopamine potentiated the spine enlargement when dopamine followed glutamatergic inputs within 0.3 – 1 s in the D1-SPNs (Yagishita et al, 2014) and similar narrow time windows of dopamine for plasticity have been shown in other studies (Wieland et al, 2015; Fisher et al, 2017). The behavioral time window, however, can also be affected by the times for sensory and reward information processing, and by other rules for plasticity (Hebb, 1949; Cassenaer and Laurent, 2012; Fremaux and Gerstner, 2015; Brea et al, 2016).…”

Section: Introductionsupporting

confidence: 84%

The minimal behavioral time window for reward conditioning in the nucleus accumbens of mice

Yamaguchi

Maeda

Sawada

et al. 2019

Preprint

View full text Add to dashboard Cite

22The temporal precision of reward-reinforcement learning is determined by the minimal time window of 23 the reward action-theoretically known as the eligibility trace. In animal studies, however, such a 24 minimal time window and its origin have not been well understood. Here, we used head-restrained mice 25 to accurately control the timing of sucrose water as an unconditioned stimulus (US); we found that the 26 reinforcement effect of the US occurred only within 1 s after a short tone of a conditioned stimulus (CS). 27The conditioning required the dopamine D1 receptor and CaMKII signaling in the nucleus accumbens 28 (NAc). The time window was not reduced by replacing CS with optogenetic stimulation of the synaptic 29 inputs to the NAc, which is in agreement with previous reports on the effective dopamine timing of NAc 30 synapses. Thus, our data suggest that the minimal reward time window is 1 s, and is formed in the NAc. 31 32 157 indicate 1 mm (left) and 20 m (right). 595

show abstract

Section: Introductionsupporting

confidence: 84%

The minimal behavioral time window for reward conditioning in the nucleus accumbens of mice

Yamaguchi

Maeda

Sawada

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…However, in their protocol dopamine activity was induced during, but not after, the preand postsynaptic pairing activity, thus demonstrating critical timing requirements but not showing silent eligibility traces outlasting glutamatergic excitation. Recently, Fisher et al (2017), reported in anaesthetized rats, that a delayed sensory stimulus paired with stimulation of the substantia nigra pars compacta modulated corticostriatal spike-timing dependent plasticity in a manner consistent with an eligibility trace. However, the pairing of presynaptic and postsynaptic activity was applied at the onset of an up state, during which the barrages of afferent activity that sustain the up state continued, so that there was no silent period free of glutamatergic excitation.…”

Section: Discussionmentioning

confidence: 95%

A silent eligibility trace enables dopamine‐dependent synaptic plasticity for reinforcement learning in the mouse striatum

Shindou

Watanabe

et al. 2018

Eur J of Neuroscience

View full text Add to dashboard Cite

Dopamine-dependent synaptic plasticity is a candidate mechanism for reinforcement learning. A silent eligibility trace - initiated by synaptic activity and transformed into synaptic strengthening by later action of dopamine - has been hypothesized to explain the retroactive effect of dopamine in reinforcing past behaviour. We tested this hypothesis by measuring time-dependent modulation of synaptic plasticity by dopamine in adult mouse striatum, using whole-cell recordings. Presynaptic activity followed by postsynaptic action potentials (pre-post) caused spike-timing-dependent long-term depression in D1-expressing neurons, but not in D2 neurons, and not if postsynaptic activity followed presynaptic activity. Subsequent experiments focused on D1 neurons. Applying a dopamine D1 receptor agonist during induction of pre-post plasticity caused long-term potentiation. This long-term potentiation was hidden by long-term depression occurring concurrently and was unmasked when long-term depression blocked an L-type calcium channel antagonist. Long-term potentiation was blocked by a Ca -permeable AMPA receptor antagonist but not by an NMDA antagonist or an L-type calcium channel antagonist. Pre-post stimulation caused transient elevation of rectification - a marker for expression of Ca -permeable AMPA receptors - for 2-4-s after stimulation. To test for an eligibility trace, dopamine was uncaged at specific time points before and after pre- and postsynaptic conjunction of activity. Dopamine caused potentiation selectively at synapses that were active 2-s before dopamine release, but not at earlier or later times. Our results provide direct evidence for a silent eligibility trace in the synapses of striatal neurons. This dopamine-timing-dependent plasticity may play a central role in reinforcement learning.

show abstract

“…Now, an existing and inactive ternary complex in AC5 regulation has consequences on how this "gate" would be opened: disinhibition from active Gα i is necessary, accompanied by stimulation from Gα olf . That is, our findings suggest that experimentally observed ACh ↓ in the striatum is likely physiologically relevant for D 1 MSNs, and both a Da ↑ and a ACh ↓ are necessary to enable synaptic potentiation (see also [46]). They can also help in interpreting the functional role of the neuromodulatory signals in the striatum.…”

Section: Relevance For Corticostriatal Synaptic Plasticitymentioning

confidence: 53%

Regulation of adenylyl cyclase 5 in striatal neurons confers the ability to detect coincident neuromodulatory signals

Bruce

Narzi

Trpevski

et al. 2019

Preprint

View full text Add to dashboard Cite

Long-term potentiation and depression of synaptic activity in response to stimuli is a key factor in reinforcement learning. Strengthening of the corticostriatal synapses depends on the second messenger cAMP, whose synthesis is catalysed by the enzyme adenylyl cyclase 5 (AC5), which is itself regulated by the stimulatory Gαolf and inhibitory Gαi proteins. AC isoforms have been suggested to act as coincidence detectors, promoting cellular responses only when convergent regulatory signals occur close in time. However, the mechanism for this is currently unclear, and seems to lie in their diverse regulation patterns. Despite attempts to isolate the ternary complex, it is not known if Gαolf and Gαi can bind to AC5 simultaneously, nor what activity the complex would have. Using protein structure-based molecular dynamics simulations, we show that this complex is stable and inactive. These simulations, along with Brownian dynamics simulations to estimate protein association rates constants, constrain a kinetic model that shows that the presence of this ternary inactive complex is crucial for AC5's ability to detect coincident signals, producing a synergistic increase in cAMP. These results reveal some of the prerequisites for corticostriatal synaptic plasticity, and explain recent experimental data on cAMP concentrations following receptor activation. Moreover, they provide insights into the regulatory mechanisms that control signal processing by different AC isoforms. Author summaryAdenylyl cyclases (ACs) are enzymes that can translate extracellular signals into the intracellular molecule cAMP, which is thus a 2 nd messenger of extracellular events. The brain expresses nine membrane-bound AC variants, and AC5 is the dominant form in the striatum. The striatum is the input stage of the basal ganglia, a brain structure involved in reward learning, i.e. the learning of behaviors that lead to rewarding stimuli (such as food, water, sugar, etc). During reward learning, cAMP production is crucial for strengthening the synapses from cortical neurons onto the striatal principal neurons, and its formation is dependent on several neuromodulatory systems such as dopamine and acetylcholine. It is, however, not understood how AC5 is activated by transient (subsecond) changes in the neuromodulatory signals. Here we combine several computational tools, from molecular dynamics and Brownian dynamics simulations to bioinformatics approaches, to inform and constrain a kinetic model of the AC5-dependent signaling system. We use this model to show how the specific molecular properties of AC5 can detect particular combinations of cooccuring transient changes in the neuromodulatory signals which thus result in a supralinear/synergistic cAMP production. Our results also provide insights into the computational capabilities of the different AC isoforms.Recent experimental data indicates that, during the resting state, much of AC5 in the D1 MSNs could be bound to Gα i due to a tonic level of acetylcholine produced by the tonic activity of the st...

show abstract

Reinforcement determines the timing dependence of corticostriatal synaptic plasticity in vivo

Cited by 88 publications

References 58 publications

The minimal behavioral time window for reward conditioning in the nucleus accumbens of mice

The minimal behavioral time window for reward conditioning in the nucleus accumbens of mice

A silent eligibility trace enables dopamine‐dependent synaptic plasticity for reinforcement learning in the mouse striatum

Regulation of adenylyl cyclase 5 in striatal neurons confers the ability to detect coincident neuromodulatory signals

Contact Info

Product

Resources

About