Reward reinforces the association between a preceding sensorimotor event and its outcome. Reinforcement learning (RL) theory and recent brain slice studies explain the delayed reward action such that synaptic activities triggered by sensorimotor events leave a synaptic eligibility trace for 1 s. The trace produces a sensitive period for reward-related dopamine to induce synaptic plasticity in the nucleus accumbens (NAc). However, the contribution of the synaptic eligibility trace to behaviour remains unclear. Here we examined a reward-sensitive period to brief pure tones with an accurate measurement of an effective timing of water reward in head-fixed Pavlovian conditioning, which depended on the plasticity-related signaling in the NAc. We found that the reward-sensitive period was within 1 s after the pure tone presentation and optogenetically-induced presynaptic activities at the NAc, showing that the short reward-sensitive period was in conformity with the synaptic eligibility trace in the NAc. These findings support the application of the synaptic eligibility trace to construct biologically plausible RL models.
22The temporal precision of reward-reinforcement learning is determined by the minimal time window of 23 the reward action-theoretically known as the eligibility trace. In animal studies, however, such a 24 minimal time window and its origin have not been well understood. Here, we used head-restrained mice 25 to accurately control the timing of sucrose water as an unconditioned stimulus (US); we found that the 26 reinforcement effect of the US occurred only within 1 s after a short tone of a conditioned stimulus (CS). 27The conditioning required the dopamine D1 receptor and CaMKII signaling in the nucleus accumbens 28 (NAc). The time window was not reduced by replacing CS with optogenetic stimulation of the synaptic 29 inputs to the NAc, which is in agreement with previous reports on the effective dopamine timing of NAc 30 synapses. Thus, our data suggest that the minimal reward time window is 1 s, and is formed in the NAc. 31 32 157 indicate 1 mm (left) and 20 m (right). 595
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.