Slow‐rising and fast‐falling dopaminergic dynamics jointly adjust negative prediction error in the ventral striatum

Abstract

The greater the reward expectations are, the more different the brain's physiological response will be. Although it is well‐documented that better‐than‐expected outcomes are encoded quantitatively via midbrain dopaminergic (DA) activity, it has been less addressed experimentally whether worse‐than‐expected outcomes are expressed quantitatively as well. We show that larger reward expectations upon unexpected reward omissions are associated with the preceding slower rise and following larger decrease (DA dip) in the DA concentration at the ventral striatum of mice. We set up a lever press task on a fixed ratio (FR) schedule requiring five lever presses as an effort for a food reward (FR5). The mice occasionally checked the food magazine without a reward before completing the task. The percentage of this premature magazine entry (PME) increased as the number of lever presses approached five, showing rising expectations with increasing proximity to task completion, and hence greater reward expectations. Fibre photometry of extracellular DA dynamics in the ventral striatum using a fluorescent protein (genetically encoded GPCR activation‐based DA sensor: GRABDA2m) revealed that the slow increase and fast decrease in DA levels around PMEs were correlated with the PME percentage, demonstrating a monotonic relationship between the DA dip amplitude and degree of expectations. Computational modelling of the lever press task implementing temporal difference errors and state transitions replicated the observed correlation between the PME frequency and DA dip amplitude in the FR5 task. Taken together, these findings indicate that the DA dip amplitude represents the degree of reward expectations monotonically, which may guide behavioural adjustment.

A Reinforcement Learning Model with Choice Traces for a Progressive Ratio Schedule

“…There were large DA increases upon reward delivery and small DA decreases upon MNPs (Fig. 3C), consistent with previous studies (Ko and Wanat, 2016;Shikano et al, 2023). A representative heatmap demonstrated a small, but clear decrease and a large increase in DA fluctuation upon MNP (Fig.…”

“…Mice were anesthetized by intraperitoneal injection of ketamine (100 mg/kg) and xylazine (10 mg/kg) before a stereotaxic surgery for adeno-associated virus (AAV) injection and implantation of an optic fiber that targeted the right VS ( Supplementary Figure 2 ). Details for surgical procedures were described in detail previously (Shikano et al, 2023). Briefly, following an incision in the scalp, a craniotomy with a diameter of 1.5 mm was created above the right VS at stereotaxic coordinates 1.1 mm anteroposterior (AP) and 1.9 mm mediolateral (ML) to the bregma.…”

“…Mice were food-restricted and trained to perform a lever-pressing operant conditioning task in FR- and PR-schedules to retrieve a food pellet, as described previously (Shikano et al, 2023; Tsutsui-Kimura et al, 2017b). Behavioral training and tests were performed under constant darkness in an aluminum operant chamber (21.6 × 17.6 × 14.0 cm; Med Associates, Fairfax, VT) housed within a sound- attenuating enclosure in a daytime.…”

“…A refractory period of 0.5 s followed each lever press before the lever was re- extended. In addition to pressing the reinforced and non-reinforced levers, mice occasionally poked into the magazine (magazine nosepoke: MNP) before making the required number of lever presses (Ko and Wanat, 2016; Shikano et al, 2023; Zhou et al, 2022). The timing of MNP was defined as the time point when the distance of the animal’s head to the center of the magazine became less than 2.5 cm.…”

“…Extracellular DA fluctuations were measured using our custom-made fiber photometric system (Natsubori et al, 2017; Shikano et al, 2023). Extracellular DA fluorescence signals were obtained by illuminating cells that expressed GRABDA2m with a 465 nm LED (8.0 ± 0.1 µW at the patch cable tip) and a 405 nm LED (8.0 ± 0.1 µW at the patch cable tip).…”