Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate

Abstract

Work in behaving primates indicates that midbrain dopamine neurons encode a prediction error, the difference between an obtained reward and the reward expected. Studies of dopamine action potential timing in the alert and anesthetized rat indicate that dopamine neurons respond in tonic and phasic modes, a distinction that has been less well characterized in the primates. We used spike train models to examine the relationship between the tonic and burst modes of activity in dopamine neurons while monkeys were performing a reinforced visuo-saccadic movement task. We studied spiking activity during four task-related intervals; two of these were intervals during which no task-related events occurred, whereas two were periods marked by task-related phasic activity. We found that dopamine neuron spike trains during the intervals when no events occurred were well described as tonic. Action potentials appeared to be independent, to occur at low frequency, and to be almost equally well described by Gaussian and Poisson-like (gamma) processes. Unlike in the rat, interspike intervals as low as 20 ms were often observed during these presumptively tonic epochs. Having identified these periods of presumptively tonic activity, we were able to quantitatively define phasic modulations (both increases and decreases in activity) during the intervals in which task-related events occurred. This analysis revealed that the phasic modulations of these neurons include both bursting, as has been described previously, and pausing. Together bursts and pauses seemed to provide a continuous, although nonlinear, representation of the theoretically defined reward prediction error of reinforcement learning.

Chronic deep brain stimulation of the rat ventral medial prefrontal cortex disrupts hippocampal–prefrontal coherence

“…The most thorough examination of this frequency band in the behaving rodent may be that of Fujisawa and Buzsáki (2011), who found that coherence between 2 and 5 Hz increased between the ventral tegmental area (VTA) and prelimbic cortex during an odor-to-place matching task, and that this oscillation was phase-coupled with theta in the dorsal hippocampus. This report extended previous observations that VTA neurons often fire action potentials at regular intervals of 4 Hz (Hyland et al, 2002;Paladini and Tepper, 1999;Bayer et al, 2007;Dzirasa et al, 2010;Kobayashi and Schultz, 2008). Hippocampal-prefrontal coherence in the 4 Hz band can also increase when dopamine is injection directly into the prefrontal cortex (Benchenane et al, 2010).…”

Computational models of reinforcement learning: the role of dopamine as a reward signal

“…This is in line with the TD model, which incorporates a discount factor to take these magnitude differences into account. Similarly, the duration of the pause of DA neuronal firing was shown to be modulated by the size of the negative reward prediction error (Bayer et al 2007). However, not all aspects of DA cell activity are modeled by the traditional TD model.…”

The role of learning-related dopamine signals in addiction vulnerability

“…because it is withheld by the experimenter or because of an error of the animal), there is a negative error in the prediction of reward (δ < 0). Dopamine neurons duly show a phasic depression in activity (Schultz et al, 1997;Tobler et al, 2003) and the duration of depressions increases with the size of the negative prediction error (Bayer et al, 2007;Mileykovskiy and Morales, 2011). Taken together, dopamine neurons seem to emit a model-free prediction error signal δ such that they are phasically more active than baseline when things are better than predicted (positive prediction error), less active than baseline when things are worse than predicted (negative prediction error) and show no change in activity when things are as good as predicted (no prediction error).…”

“…By contrast, the dopamine neurons of monkeys that have learned to predict reward well show conditioned stimulus responses indicative of learning in an asymmetrically rewarded saccade task . In behavioral situations with contingencies changing about every 100 trials, dopamine neurons code the difference between current reward and reward history weighted by the last six to seven trials (Bayer et al, 2007). The occurrence of reward or reward prediction (positive prediction error) or their omission (negative prediction error) activates or depresses dopamine neurons in an inverse monotonic function of probability, such that the more unpredicted the event the stronger the response (de Lafuente and Romo, 2011;Enomoto et al, 2011;Fiorillo et al, 2003;Matsumoto and Hikosaka, 2009;Morris et al, 2006;Nakahara et al, 2004;Nomoto et al, 2010;Oyama et al, 2010;Satoh et al, 2003).…”