We recorded the activity of midbrain dopamine neurons in an instrumental conditioning task in which monkeys made a series of behavioral decisions on the basis of distinct reward expectations. Dopamine neurons responded to the first visual cue that appeared in each trial [conditioned stimulus (CS)] through which monkeys initiated trial for decision while expecting trial-specific reward probability and volume. The magnitude of neuronal responses to the CS was approximately proportional to reward expectations but with considerable discrepancy. In contrast, CS responses appear to represent motivational properties, because their magnitude at trials with identical reward expectation had significant negative correlation with reaction times of the animal after the CS. Dopamine neurons also responded to reinforcers that occurred after behavioral decisions, and the responses precisely encoded positive and negative reward expectation errors (REEs). The gain of coding REEs by spike frequency increased during learning act-outcome contingencies through a few months of task training, whereas coding of motivational properties remained consistent during the learning. We found that the magnitude of CS responses was positively correlated with that of reinforcers, suggesting a modulation of the effectiveness of REEs as a teaching signal by motivation. For instance, rate of learning could be faster when animals are motivated, whereas it could be slower when less motivated, even at identical REEs. Therefore, the dual correlated coding of motivation and REEs suggested the involvement of the dopamine system, both in reinforcement in more elaborate ways than currently proposed and in motivational function in reward-based decision-making and learning.

Midbrain dopamine neurons signal reward value, their prediction error, and the salience of events. If they play a critical role in achieving specific distant goals, long-term future rewards should also be encoded as suggested in reinforcement learning theories. Here, we address this experimentally untested issue. We recorded 185 dopamine neurons in three monkeys that performed a multistep choice task in which they explored a reward target among alternatives and then exploited that knowledge to receive one or two additional rewards by choosing the same target in a set of subsequent trials. An analysis of anticipatory licking for reward water indicated that the monkeys did not anticipate an immediately expected reward in individual trials; rather, they anticipated the sum of immediate and multiple future rewards. In accordance with this behavioral observation, the dopamine responses to the start cues and reinforcer beeps reflected the expected values of the multiple future rewards and their errors, respectively. More specifically, when monkeys learned the multistep choice task over the course of several weeks, the responses of dopamine neurons encoded the sum of the immediate and expected multiple future rewards. The dopamine responses were quantitatively predicted by theoretical descriptions of the value function with time discounting in reinforcement learning. These findings demonstrate that dopamine neurons learn to encode the long-term value of multiple future rewards with distant rewards discounted.decision making | basal ganglia | temporal difference learning | primate

Fluoro-Jade C (FJC) staining is widely used for the specific detection of all degenerating mature neurons, including apoptotic, necrotic, and autophagic cells. However, whether FJC staining can detect degenerating immature neurons and neural stem/precursor cells remains unclear. In addition, some conflicting studies have shown that FJC and its ancestral dyes, Fluoro-Jade (FJ) and FJB, can label resting/activated astrocytes and microglia. In the present study, we examined the validity of FJC staining for the detection of neuronal cells in adult and embryonic mouse brains under normal and injured conditions. In the adult rodent subventricular zone-rostral migratory stream-olfactory bulb system, apoptosis associated with neurogenesis occurs under normal conditions. Using this system, we detected FCJ positive (+) cells, some of which were doublecortin (DCX)(+) neuroblasts, in addition to neuronal nuclei (NeuN)(+) mature neurons. FJC negative (-) apoptotic cells expressing activated Caspase 3 were also observed, and a small number of FJC(+)/ionized calcium-binding adaptor 1 (Iba1)(+) microglia and FJC(+)/glial fibrillary acidic protein (GFAP)(+) astrocytes were observed in the normal brain. Next, we analyzed embryonic brains, in which the apoptosis of neural stem/precursor cells was induced by the administration of N-ethyl-N-nitrosourea (ENU) or ethanol at embryonic day 14 or 10, respectively. In those brains, FJC(+) neural stem/precursor cells and neuroepithelial cells expressing SRY-related HMG-box 2 (Sox2) were observed. Surprisingly degenerating mesenchymal cells were also FJC(+). The present study indicates that FJC is a reliable marker for degenerating neuronal cells during all differentiation stages. However, FJC could also label degenerating non-neuronal cells under some conditions. Highlights• Fluoro-Jade C can label degenerating immature neurons during adult neurogenesis under normal conditions.• Fluoro-Jade C can label neural stem/precursor cells in apoptosis-induced embryonic brains.• Some degenerating microglia and astrocytes can be Fluoro-Jade C positive under some conditions.• Apoptotic mesenchymal cells induced by the administration of a toxic dose of ethanol were also Fluoro-Jade C positive. Keywords cell death, neural stem/precursor cells, adult neurogenesis, embryonic brain , glia, mesenchymal cells 4 Abbreviations Cas3, activated Caspase3 DCX, doublecortin ENU, N-ethyl-N-nitrosourea FJ, Fluoro-Jade GABA, gamma-aminobutyric acid GAD67, glutamic acid decarboxylase 67 GFAP, glial fibrillary acidic protein GFP, green fluorescent protein

To study roles of cortico-basal ganglia loops in action planning, we examined interactions between the activities of simultaneously recorded neurons in the striatum of monkeys performing sequence motor tasks by cross-correlation analysis. Serial activation occurred between projection neurons in a motor sequence-dependent manner, and was in the direction of a neuron encoding an early event in the sequence to a neuron encoding the same event or later, but closer event to the reward. Synchronous activation occurred between pairs of interneurons. The serial activation seems to originate through the cortico-basal ganglia loops, because projection neurons are inhibitory. We propose that the task-dependent serial and synchronous activation of striate neurons may be a neural substrate for goal-directed planning through the basal ganglia.