Causal evidence supporting the proposal that dopamine transients function as a temporal difference prediction error

Abstract

Reward-evoked dopamine is well-established as a prediction error. However the central tenet of temporal difference accounts -that similar transients evoked by reward-predictive cues also function as errors -remains untested. To address this, we used two phenomena, second-order conditioning and blocking, in order to examine the role of dopamine in prediction error versus reward prediction. We show that optogenetically-shunting dopamine activity at the start of a reward-predicting cue prevents second-order conditioning without affecting blocking. These results support temporal difference accounts by providing causal evidence that cue-evoked dopamine transients function as prediction errors.

Temporally restricted dopaminergic control of reward-conditioned movements

“…Additionally, in our study we did not explicitly check for RPE coding, and thus cannot rule out that DA neurons represent additional or different information 16 , 22 , 46 . Our results leave open the possibility that pre-reward DA signals are important for functions that were not studied here, such as salience 46 , time perception 47 , or second order conditioning 48 .…”

“…On the other hand, several studies show a strong reward response in trained animals 20,22,24 , and one of these suggests that the pre-and postreward DA signals evolve independently 20 . Additionally, in our study we did not explicitly check for RPE coding, and thus cannot rule out that DA neurons represent additional or different information 16, 22, 46 . Our results leave open the possibility that pre-reward DA signals are important for functions that were not studied here, such as salience 46 , time perception 47 , or second order conditioning 48 .…”

Dopamine Signaling Is Critical for Supporting Cue-Driven Behavioral Control

“…Indeed, the ventral tegmental area (VTA) DA signal has been linked to one of the most fundamental teaching algorithms in learning, namely error-correction (Rescorla and Wagner, 1972;Sutton and Barto, 1990;Schultz et al, 1997). Studies has provided both correlational (Waelti et al, 2001) and causal evidence for the role of VTA DA in associative learning, with optogenetic activation or inhibition of VTA DA neurons enhancing or disrupting learning, respectively (Steinberg et al, 2013;Chang et al, 2016;Keiflin et al, 2017;Maes et al, 2019). Beyond this, DA has also been linked to a number of reward-dependent behavioural and psychological constructs including action initiation, vigor, effort, motivation, incentive salience and incentive learning (Berridge, 2007;Wassum et al, 2013;Ostlund et al, 2014;Hamid et al, 2016;Ko and Wanat, 2016).…”