Dopamine transients are sufficient and necessary for acquisition of model-based associations

Abstract

Associative learning is driven by prediction errors. Dopamine transients correlate with these errors, which current interpretations limit to endowing cues with a scalar quantity reflecting the value of future rewards. Here, we tested whether dopamine might act more broadly to support learning of an associative model of the environment. Using sensory preconditioning, we show that prediction errors underlying stimulus-stimulus learning can be blocked behaviorally and reinstated by optogenetically activating dopamine neurons. We further show that suppressing the firing of these neurons across t transition prevents normal stimulus-stimulus learning. These results establish that the acquisition of model-based information about transitions between non-rewarding events is also driven by prediction errors, and that contrary to existing canon, dopamine transients are both sufficient and necessary to support this type of learning. Our findings open new possibilities for how these biological signals might support associative learning in the mammalian brain in these and other contexts.

Optogenetic Blockade of Dopamine Transients Prevents Learning Induced by Changes in Reward Features

“…However, while these causal studies show that dopamine transients (1–2s increases or decreases) are sufficient to drive learning, they generally do not address whether they are necessary, since they either do not attempt to block the normal physiological signal or they do so with a manipulation capable of acting independently to drive learning. Further, these causal studies do not address whether dopamine transients only mediate learning in response to cached-value errors or whether these signals may also facilitate learning about rewards in settings where errors in predicting cached value are not responsible, as has been recently suggested for neutral cues [23]. …”

“…While such firing changes have been explained as salience or a value bonus tied to novelty [31, 34], they might be viewed as an error evoked by the change in sensory input. Accordingly, we have recently found that dopamine transients are sufficient and likely necessary for learning to associate neutral cues [23]. Their role in this function was not tied in any obvious way to salience or changes in the cues’ associability.…”

Dopamine Neurons Respond to Errors in the Prediction of Sensory Features of Expected Rewards

“…For example, cached value errors would not occur in situations in which value remains the same but the specific properties of the predicted event change. Such learning occurs commonly in our daily lives and can be isolated in experimental procedures such as identity or transreinforcer unblocking and sensory preconditioning (Brogden, 1939; Burke et al, 2008; Jones et al, 2012; Rescorla, 1999), the latter recently shown to depend on dopamine transients (Sharpe et al, 2017). The existence of these value-neutral forms of learning (and the fact that they can be blocked) suggests that there must be some mechanism for signaling the underlying sensory or identity prediction errors.…”

A Perceptual Inference Mechanism for Hallucinations Linked to Striatal Dopamine

“…Beyond their role in reinforcement learning [9–11], neuromodulators including dopamine have been proposed to contribute to experience-dependent sensory learning [12–14]. Specifically, Bayesian models posit that perception results from an optimal integration of bottom-up sensory evidence and top-down sensory predictions or priors [15].…”