Temporally Extended Dopamine Responses to Perceptually Demanding Reward-Predictive Stimuli

Abstract

Midbrain dopamine neurons respond to reward-predictive stimuli. In the natural environment reward-predictive stimuli are often perceptually complicated. Thus, to discriminate one stimulus from another, elaborate sensory processing is necessary. Given that previous studies have used simpler types of reward-predictive stimuli, it has yet to be clear whether and, if so, how dopamine neurons obtain reward information from perceptually complicated stimuli. To investigate this, we recorded the activities of monkey dopamine neurons while they were performing discrimination between two coherent motion directions in random-dot motion stimuli. These coherent directions were paired with different magnitudes of reward. We found that dopamine neurons showed reward-predictive responses to random-dot motion stimuli. Moreover, dopamine neurons showed temporally extended activity correlated with changes in reward prediction (i.e., reward prediction error) from coarse to fine scales between initial motion detection and subsequent motion discrimination phases. Noticeably, dopamine reward-predictive responses became differential in a later phase than previously reported. This response pattern was consistent with the time course of processing required for the estimation of expected reward value that parallels the motion direction discrimination processing. The results demonstrate that dopamine neurons are able to reflect the reward value of perceptually complicated stimuli, and suggest that dopamine neurons use the moment-to-moment reward prediction associated with environmental stimuli to compute a reward prediction error.

Opponent actor learning (OpAL): Modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive.

“…Specifically, phasic firing of midbrain dopamine neurons convey reward prediction errors that facilitate plasticity in the striatum (Montague, Dayan, & Sejnowski, 1996;Schultz, 1997). Many studies have since provided strong support for this notion (Arias-Carrión, Stamelou, Murillo-Rodríguez, Menéndez-González, & Pöppel, 2010;Bayer & Glimcher, 2005;Bayer, Lau, & Glimcher, 2007;Nakahara, Itoh, Kawagoe, Takikawa, & Nikosaka, 2004;Nomoto, Schultz, Watanabe, & Sakagami, 2010). Reinforcement learning models have been routinely used to account for dopaminergic modulation of behavioral and neural signals during learning tasks (Frank, Moustafa, Haughey, Curran, & Hutchison, 2007;McClure, Daw, & Read Montague, 2003;O'Doherty et al, 2004;Pessiglione, Seymour, Flandin, Dolan, & Frith, 2006;Samejima, Ueda, Doya, & Kimura, 2005;Schönberg, Daw, Joel, & O'Doherty, 2007).…”

The role of learning-related dopamine signals in addiction vulnerability

“…In behavioral situations with contingencies changing about every 100 trials, dopamine neurons code the difference between current reward and reward history weighted by the last six to seven trials (Bayer et al, 2007). The occurrence of reward or reward prediction (positive prediction error) or their omission (negative prediction error) activates or depresses dopamine neurons in an inverse monotonic function of probability, such that the more unpredicted the event the stronger the response (de Lafuente and Romo, 2011;Enomoto et al, 2011;Fiorillo et al, 2003;Matsumoto and Hikosaka, 2009;Morris et al, 2006;Nakahara et al, 2004;Nomoto et al, 2010;Oyama et al, 2010;Satoh et al, 2003). Enomoto et al (2011) attempted to directly address whether the phasic dopamine response reflect the total future reward, as opposed to just the immediate reward.…”

Sensory regulation of dopaminergic cell activity: Phenomenology, circuitry and function

“…The truly discriminatory phase of the DA response comes after an earlier, more invariant response feature (Hudgins et al, 2009;Nomoto et al, 2010). Given the rather primitive processing capabilities of the SC, the natural assumption would be that the shorter latency component of the DA response is of collicular origin whilst the longer latency discriminatory phase has its origin elsewhere in the brain (Hudgins et al, 2009).…”

“…Both phases are comparatively short. The second phase of the response of DA neurons to complex visual stimuli reported in the monkey has a peak latency of around 160 ms (Morris et al 2004;Hudgins et al 2009;Nomoto et al 2010). Given that saccades take 180-200 ms to initiate (Becker, 1989) and then last for 3-100 ms/degree depending on amplitude (Bahill et al, 1975), both the second phase and the initial shorter latency phase of the response to complex visual stimuli are pre-saccadic under conditions of experimenter-driven stimulus presentation, i.e.…”

“…they occur before the animal has had a chance to move its eyes to foveate the stimulus concerned. Indeed, the second phase of the visual response described by Nomoto et al (2010) is often co-incident with saccade generation.…”