Sequential delay and probability discounting tasks in mice reveal anchoring effects partially attributable to decision noise

Rojas, Gerardo; Curry-Pochy, Lisa S.; Chen, Cathy S.; Heller, Abigail T.; Grissom, Nicola M.

doi:10.1016/j.bbr.2022.113951

Cited by 5 publications

(5 citation statements)

References 62 publications

(98 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5 C ). These data were fitted with the hyperbolic discount function

, [ t for time, (s)], yielding discounting coefficient ( k ), values of

0.48 0.019 029 0.014 0.092 0.02

) (mean

SD,

), for the three reward magnitudes (sucrose, 8, 5, 2.2 μL, respectively) which are comparable to values reported in mice, 0.3 ( 48 , 49 ) and 0.1–0.5 ( 49 ) ( Fig. 5 C ).…”

Section: Resultssupporting

confidence: 59%

“…Since the subjective value of a future reward is determined by the amplitude of the DAergic CR to the immediately available predictive cue, delay discounting implies that the CR amplitude will diminish with increasing reward delay. This expectation is confirmed by the similarity of the ratio of the reward equivalents of the dopaminergic CR and UCR firing rates (~0.62) ( 2 ) and the ratio of subjective reward values of the immediately available and delayed rewards measured in behavioral experiments (0.57–0.69) ( 48 , 49 ). Within the TDL model, a reduction of the CR would result from a time-dependent amplification of the amplitude of

during the CS-UCS interval.…”

Section: Resultsmentioning

confidence: 65%

See 1 more Smart Citation

Neuronal implementation of the temporal difference learning algorithm in the midbrain dopaminergic system

Stetsenko,

Koos

2023

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

The temporal difference learning (TDL) algorithm has been essential to conceptualizing the role of dopamine in reinforcement learning (RL). Despite its theoretical importance, it remains unknown whether a neuronal implementation of this algorithm exists in the brain. Here, we provide an interpretation of the recently described signaling properties of ventral tegmental area (VTA) GABAergic neurons and show that a circuitry of these neurons implements the TDL algorithm. Specifically, we identified the neuronal mechanism of three key components of the TDL model: a sustained state value signal encoded by an afferent input to the VTA, a temporal differentiation circuit formed by two types of VTA GABAergic neurons the combined output of which computes momentary reward prediction (RP) as the derivative of the state value, and the computation of reward prediction errors (RPEs) in dopamine neurons utilizing the output of the differentiation circuit. Using computational methods, we also show that this mechanism is optimally adapted to the biophysics of RPE signaling in dopamine neurons, mechanistically links the emergence of conditioned reinforcement to RP, and can naturally account for the temporal discounting of reinforcement. Elucidating the implementation of the TDL algorithm may further the investigation of RL in biological and artificial systems.

show abstract

“…5 C ). These data were fitted with the hyperbolic discount function

, [ t for time, (s)], yielding discounting coefficient ( k ), values of

0.48 0.019 029 0.014 0.092 0.02

) (mean

SD,

), for the three reward magnitudes (sucrose, 8, 5, 2.2 μL, respectively) which are comparable to values reported in mice, 0.3 ( 48 , 49 ) and 0.1–0.5 ( 49 ) ( Fig. 5 C ).…”

Section: Resultssupporting

confidence: 59%

during the CS-UCS interval.…”

Section: Resultsmentioning

confidence: 65%

Neuronal implementation of the temporal difference learning algorithm in the midbrain dopaminergic system

Stetsenko,

Koos

2023

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…Training was similar to our previous experiment (Rojas et al, 2022). Mice began with magazine training for 3 days and moved on to training schedules.…”

Section: Methodsmentioning

confidence: 95%

“…MIce underwent sequences of Worsening and Improving schedules as previously described (Rojas et al, 2022). One deviation from the previous study is that mice did not receive forced-choice trials in order to prioritize completion of delay blocks.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Differential processing of delay versus uncertainty in male but not female 16p11.2 hemideletion mice

Rojas,

Heller,

Grissom

2023

Preprint

Self Cite

View full text Add to dashboard Cite

Neurodevelopmental disorders are associated with differences in learning and motivation that can influence executive function, including behavioral flexibility and decision making. 16p11.2 hemideletion is a chromosomal copy number variant that is linked to neurodevelopmental disorders. 16p11.2 hemideletion in mice has been previously found to produce male-biased changes in reward learning, but the link between this and altered flexible decision making is poorly understood. We challenged 16p11.2 hemideletion mice with two reward-guided decision making tasks assessing flexible decision making under cost, delay and probability discounting. Both tasks elicited long-term changes in flexible decision making that separated 16p11.2 hemideletion males from wildtype males. In delay discounting,16p11.2 hemideletion males had a stronger, less flexible preference for the large reward at long delays, and this effect was reduced as wildtype males adjusted their preference to match that of the hemideletion males. In probability discounting, 16p11.2 hemideletion males initially had a similar preference for seeking improbable large rewards as did wildtype males, but over time began to prefer certainty to a greater extent than did wildtype males. Female mice discounted similarly for delayed or risky rewards regardless of the presence of the copy number variant. We have previously seen that male 16p11.2 hemideletion mice commit fewer nonreinforced responses than male wildtype mice in an operant setting, which we replicate here in delay discounting, while the introduction of risky rewards eliminates genotype differences in nonreinforced responses. Overall these data suggest that 16p11.2 hemideletion in males leads to differential processing of costs of delay versus inconsistency, with greater aversion to uncertainty than delays, and greater behavioral control by cues that consistently predict an outcome.

show abstract

Effects of delay sequence in a delay discounting task

Macías-Navarrete,

dos Santos

2024

Behavioural Processes

View full text Add to dashboard Cite

Sequential delay and probability discounting tasks in mice reveal anchoring effects partially attributable to decision noise

Cited by 5 publications

References 62 publications

Neuronal implementation of the temporal difference learning algorithm in the midbrain dopaminergic system

Neuronal implementation of the temporal difference learning algorithm in the midbrain dopaminergic system

Differential processing of delay versus uncertainty in male but not female 16p11.2 hemideletion mice

Effects of delay sequence in a delay discounting task

Contact Info

Product

Resources

About