Midbrain Dopaminergic Neurons and Striatal Cholinergic Interneurons Encode the Difference between Reward and Aversive Events at Different Epochs of Probabilistic Classical Conditioning Trials

Abstract

Midbrain dopaminergic neurons (DANs) typically increase their discharge rate in response to appetitive predictive cues and outcomes, whereas striatal cholinergic tonically active interneurons (TANs) decrease their rate. This may indicate that the activity of TANs and DANs is negatively correlated and that TANs can broaden the basal ganglia reinforcement teaching signal, for instance by encoding worse than predicted events. We studied the activity of 106 DANs and 180 TANs of two monkeys recorded during the performance of a classical conditioning task with cues predicting the probability of food, neutral, and air puff outcomes. DANs responded to all cues with elevations of discharge rate, whereas TANs depressed their discharge rate. Nevertheless, although dopaminergic responses to appetitive cues were larger than their responses to neutral or aversive cues, the TAN responses were more similar. Both TANs and DANs responded faster to an air puff than to a food outcome; however, DANs responded with a discharge elevation, whereas the TAN responses included major negative and positive deflections. Finally, food versus air puff omission was better encoded by TANs. In terms of the activity of single neurons with distinct responses to the different behavioral events, both DANs and TANs were more strongly modulated by reward than by aversive related events and better reflected the probability of reward than aversive outcome. Thus, TANs and DANs encode the task episodes differentially. The DANs encode mainly the cue and outcome delivery, whereas the TANs mainly encode outcome delivery and omission at termination of the behavioral trial episode.

Neurogenetics and Pharmacology of Learning, Motivation, and Cognition

“…Across a range of species from rats to humans, dopamine cells burst fire in response to positive 'prediction errors' (events that are better than expected), whereas these same cells pause in response to negative prediction errors (events that are worse than expected) (Schultz et al, 1997;Bayer and Glimcher, 2005;Roesch et al, 2007;Joshua et al, 2006;Pan et al, 2008;Zaghloul et al, 2009). A key assumption of reinforcement learning models is that these bursts and dips act as a 'teaching signal' by modifying synaptic plasticity in target structures Wickens et al, 2003;Frank, 2005).…”

Sensory regulation of dopaminergic cell activity: Phenomenology, circuitry and function

“…Auditory and simple visual stimuli associated with aversive air puffs to the hand also activate some midbrain DA neurons although they inhibit others (Mirenowicz and Schultz, 1996). However, most recent studies in the monkey have abandoned the use of simple visual stimuli in favour of coloured, geometrically complex visual stimuli, to which midbrain DA neurons respond with activation when the stimuli are paired with reward (Waelti et al, 2001;Morris et al, 2004;Fiorillo et al, 2003;Tobler et al, 2005;Joshua et al, 2008;Hudgens et al, 2009) or aversive outcomes (Fiorillo et al, 2013a). Under rewarding conditions, responses to complex visual stimuli are modulated by the probability and magnitude of the reward they predict (Waelti et al, 2001;Morris et al, 2004;Fiorillo et al, 2003;Tobler et al, 2005;Joshua et al, 2008;Hudgens et al, 2009).…”

“…However, most recent studies in the monkey have abandoned the use of simple visual stimuli in favour of coloured, geometrically complex visual stimuli, to which midbrain DA neurons respond with activation when the stimuli are paired with reward (Waelti et al, 2001;Morris et al, 2004;Fiorillo et al, 2003;Tobler et al, 2005;Joshua et al, 2008;Hudgens et al, 2009) or aversive outcomes (Fiorillo et al, 2013a). Under rewarding conditions, responses to complex visual stimuli are modulated by the probability and magnitude of the reward they predict (Waelti et al, 2001;Morris et al, 2004;Fiorillo et al, 2003;Tobler et al, 2005;Joshua et al, 2008;Hudgens et al, 2009). Responses either follow unpredicted experimenter-driven stimulus presentation within task (Waelti et al, 2001;Morris et al, 2004;Fiorillo et al, 2003;Tobler et al, 2005;Joshua et al, 2008;Hudgens et al, 2009), or when the animal itself 'discovers' the stimulus during visual search (Matsumoto, 2013).…”

“…Under rewarding conditions, responses to complex visual stimuli are modulated by the probability and magnitude of the reward they predict (Waelti et al, 2001;Morris et al, 2004;Fiorillo et al, 2003;Tobler et al, 2005;Joshua et al, 2008;Hudgens et al, 2009). Responses either follow unpredicted experimenter-driven stimulus presentation within task (Waelti et al, 2001;Morris et al, 2004;Fiorillo et al, 2003;Tobler et al, 2005;Joshua et al, 2008;Hudgens et al, 2009), or when the animal itself 'discovers' the stimulus during visual search (Matsumoto, 2013). …”

The Aversive Agent Lithium Chloride Suppresses Phasic Dopamine Release Through Central GLP-1 Receptors

“…Phasic increases in the firing of midbrain ventral tegmental (VTA) dopamine neurons and resulting phasic increases in extracellular nucleus accumbens (NAc) dopamine concentration occur both spontaneously and in response to either unconditioned primary rewards or conditioned predictors of reward (Cohen et al, 2012;Joshua et al, 2008;Matsumoto and Hikosaka, 2009;Owesson-White et al, 2012;Roitman et al, 2004;Schultz, 1998;Sombers et al, 2009;Zweifel et al, 2009). These phasic increases are both necessary and sufficient for positive reinforcement and associative learning (Steinberg et al, 2013;Tsai et al, 2009), supporting a mechanism by which rewarding stimuli reinforce approach behaviors necessary for survival (eg procuring food).…”