Ventral Tegmental Dopamine Neurons Participate in Reward Identity Predictions

Abstract

SUMMARY Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) encode reward prediction errors (RPEs) and are proposed to mediate error-driven learning. However the learning strategy engaged by DA-RPEs remains controversial. RPEs might imbue cue/actions with pure value, independently of representations of their associated outcome. Alternatively, RPEs might promote learning about the sensory features (the identity) of the rewarding outcome. Here we show that although both VTA and SNc DA neuron activation reinforces instrumental responding, only VTA DA neuron activation during consumption of expected sucrose reward restores error-driven learning and promotes formation of a new cue➔sucrose association. Critically, expression of VTA DA-dependent Pavlovian associations is abolished following sucrose devaluation, a signature of identity-based learning. These findings reveal that activation of VTA- or SNc-DA neurons engages largely dissociable learning processes with VTA-DA neurons capable of participating outcome-specific predictive learning, while the role of SNc-DA neurons appears limited to reinforcement of instrumental responses.

Dopamine neuron ensembles signal the content of sensory prediction errors

“…However, unlike current canon, this proposal also easily explains why dopamine neurons are often phasically active in settings where value errors were not anticipated a priori, at least by the experimenters, such as when novel cues or even information is first presented (Bromberg-Martin and Hikosaka, 2009;Horvitz, 2000;Horvitz et al, 1997;Kakade and Dayan, 2002), or even in response to violations in beliefs or auditory expectations (Glascher et al, 2010;Gold et al, 2019;Iglesias et al, 2013;Schwartenbeck et al, 2016). Further, it provides a neural basis for recent demonstrations that dopamine transients are necessary for learning that cannot be easily accounted for by classic reinforcement learning mechanisms Keiflin et al, 2019;Sharpe et al, 2017).…”

“…Such sensory prediction errors would be useful for learning detailed information about the relationships between real-world events (Gardner et al, 2018;Howard and Kahnt, 2018;Langdon et al, 2017;Takahashi et al, 2017). Indeed, dopamine transients facilitate learning such relationships, independent of value, when they are appropriately positioned to mimic endogenous errors Keiflin et al, 2019;Sharpe et al, 2017). Yet dopaminergic sensory prediction error signals do not seem to encode the content of the mis-predicted event, either at the level of individual neurons or summed across populations (Howard and Kahnt, 2018;Takahashi et al, 2017).…”

Causal evidence supporting the proposal that dopamine transients function as a temporal difference prediction error

“…This is a noteworthy caveat, since temporal difference errors can be limited to representing information about value 2,3 or they can be construed more broadly as representing errors in predicting other value-neutral information 2,10 . Recent studies using sensory preconditioning and reinforcer devaluation provide evidence that dopamine transients support learning that is orthogonal to value and in line with the latter account [9][10][11] . Here we used second-order conditioning, which has been proposed to rely on an associative structure that bypasses the representation of the outcome and links a stimulus and a response 12 .…”

In vivo functional diversity of midbrain dopamine neurons within identified axonal projections

“…The functional properties and the emerging diversity of the midbrain dopamine (DA) system have been extensively studied across many different biological scales (Schultz, 2015;Watabe-Uchida et al, 2017). These differences range from the single-cell level, including diverging gene expression profiles (Kramer et al, 2018;Nichterwitz et al, 2016;Poulin et al, 2018;Saunders et al, 2018a;Tiklova et al, 2019), cellular and biophysical properties Lammel et al, 2008;Tarfa et al, 2017), as well as neurotransmitter co-release (Chuhma et al, 2018;Kim et al, 2015b;Tritsch et al, 2012;Zhang et al, 2015), up to different neural circuit affiliations (Beier et al, 2019;Beier et al, 2015;Lammel et al, 2012;Lerner et al, 2015;Menegas et al, 2015;Ogawa et al, 2014;Tian et al, 2016;Watabe-Uchida et al, 2012) and selective task engagement of DA subpopulations in awake behaving rodents (Dautan et al, 2016;de Jong et al, 2019;Duvarci et al, 2018;Gunaydin et al, 2014;Howe and Dombeck, 2016;Keiflin et al, 2019;Lammel et al, 2012;Matthews et al, 2016;Menegas et al, 2018;Menegas et al, 2017;Parker et al, 2016;Patriarchi et al, 2018;Saunders et al, 2018b;Vander Weele et al, 2018;Yang et al, 2018) and non-human primates …”