Effects of dopamine depletion from the caudate-putamen and nucleus accumbens septi on the acquisition and performance of a conditional discrimination task

The authors present their primary value learned value (PVLV) model for understanding the rewardpredictive firing properties of dopamine (DA) neurons as an alternative to the temporal-differences (TD) algorithm. PVLV is more directly related to underlying biology and is also more robust to variability in the environment. The primary value (PV) system controls performance and learning during primary rewards, whereas the learned value (LV) system learns about conditioned stimuli. The PV system is essentially the Rescorla-Wagner/delta-rule and comprises the neurons in the ventral striatum/nucleus accumbens that inhibit DA cells. The LV system comprises the neurons in the central nucleus of the amygdala that excite DA cells. The authors show that the PVLV model can account for critical aspects of the DA firing data, making a number of clear predictions about lesion effects, several of which are consistent with existing data. For example, first-and second-order conditioning can be anatomically dissociated, which is consistent with PVLV and not TD. Overall, the model provides a biologically plausible framework for understanding the neural basis of reward learning.Keywords: basal ganglia, dopamine, reinforcement learning, Pavlovian conditioning, computational modeling An important and longstanding challenge for both the cognitive neuroscience and artificial intelligence communities has been to develop an adequate understanding (and a correspondingly robust model) of Pavlovian learning. Such a model should account for the full range of signature findings in the rich literature on this phenomenon. Pavlovian conditioning refers to the ability of previously neutral stimuli that reliably co-occur with primary rewards to elicit new conditioned behaviors and to take on reward value themselves (e.g., Pavlov's famous case of the bell signaling food for hungry dogs; Pavlov, 1927).Pavlovian conditioning is distinguished from instrumental conditioning in that the latter involves the learning of new behaviors that are reliably associated with reward, either first order (US), or second order (CS). Although Pavlovian conditioning also involves behaviors (conditioned and unconditioned responses), reward delivery is not contingent on behavior but is instead reliably paired with a stimulus regardless of behavior. In contrast, instrumental conditioning explicitly makes reward contingent on a particular "operant" or "instrumental" response. Both stimulus-reward (Pavlovian) and stimulus-response-reward (instrumental) associations, however, are thought to be trained by the same phasic dopamine signal that occurs at the time of primary reward (US) as described below. In practice, the distinction is often blurry as the two types of conditioning interact (e.g., second-order instrumental conditioning and so-called Pavlovian instrumental transfer effects).The dominant theoretical perspective for both Pavlovian and instrumental conditioning since the seminal Rescorla and Wagner (1972) model, is that learning is based on the discrepancy between act...

show abstract

“…There is some data consistent with this prediction (Ferry, Lu, & Price, 2000;Robbins, Giardini, Jones, Reading, & Sahakian, 1990).…”

Section: Testable Predictions Of the Modelsupporting

confidence: 52%

PVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm.

O’Reilly

Frank

Hazy

et al. 2007

Behavioral Neuroscience

111

115

View full text Add to dashboard Cite

show abstract

“…The poor performance of the Mn group on the FR 1 task may reflect an associative deficit, because the CPP data suggest that reward processes were unaffected by Mn exposure, while performance on the progressive ratio schedule indicated that Mn-treated rats did not have motivational or motoric deficits. Both the striatum and nucleus accumbens are important for operant responding, with dopamine transmission in the striatum being necessary for acquisition of procedural tasks and habit formation (Robbins et al, 1990;Featherstone and McDonald, 2004a;Yin et al, 2004;Faure et al, 2005), whereas accumbal dopamine release is important for sustained goal-directed behaviors (for reviews, see Salamone and Correa, 2002;Salamone et al, 2003). 6-Hydroxydopamine (6-OHDA) lesions of the nucleus accumbens disrupt performance on high, but not low, FR schedules (Aberman and Salamone, 1999), suggesting that Mn-induced changes in accumbal functioning were not responsible for the performance decrement on our FR 1 task.…”

Section: Discussionmentioning

confidence: 99%

“…6-Hydroxydopamine (6-OHDA) lesions of the nucleus accumbens disrupt performance on high, but not low, FR schedules (Aberman and Salamone, 1999), suggesting that Mn-induced changes in accumbal functioning were not responsible for the performance decrement on our FR 1 task. Instead, the striatum is suspect, since 6-OHDA and quinolinic acid lesions of the dorsolateral striatum disrupt acquisition of even simple operant tasks (Robbins et al, 1990;McDonald, 2004a, 2005). Because only FR 1 responding was affected in the present study, the most parsimonious explanation is that Mn-induced alterations in striatal functioning may have caused a modest impairment in stimulus-response learning.…”

Section: Discussionmentioning

confidence: 99%

Postnatal manganese exposure alters dopamine transporter function in adult rats: Potential impact on nonassociative and associative processes

McDougall¹,

Reichel²,

Farley³

et al. 2008

Neuroscience

View full text Add to dashboard Cite

In the present study, we examined whether exposing rats to a high-dose regimen of manganese chloride (Mn) during the postnatal period would depress presynaptic dopamine functioning and alter nonassociative and associative behaviors. To this end, rats were given oral supplements of Mn (750 μg/day) on postnatal days (PD) 1-21. On PD 90, dopamine transporter (DAT) immunoreactivity and [ 3 H]dopamine uptake were assayed in the striatum and nucleus accumbens, while in vivo microdialysis was used to measure dopamine efflux in the same brain regions. The effects of postnatal Mn exposure on nigrostriatal functioning were evaluated by assessing rotorod performance and amphetamine-induced stereotypy in adulthood. In terms of associative processes, both cocaineinduced conditioned place preference (CPP) and sucrose-reinforced operant responding were examined. Results showed that postnatal Mn exposure caused persistent declines in DAT protein expression and [ 3 H]dopamine uptake in the striatum and nucleus accumbens, as well as long-term reductions in striatal dopamine efflux. Rotorod performance did not differ according to exposure condition, however Mn-exposed rats did exhibit substantially more amphetamine-induced stereotypy than vehicle controls. Mn exposure did not alter performance on any aspect of the CPP task (preference, extinction, or reinstatement testing), nor did Mn affect progressive ratio responding (a measure of motivation). Interestingly, acquisition of a fixed ratio task was impaired in Mn-exposed rats, suggesting a deficit in procedural learning. In sum, these results indicate that postnatal Mn exposure causes persistent declines in various indices of presynaptic dopaminergic functioning. Mninduced alterations in striatal functioning may have long-term impact on associative and nonassociative behavior.

show abstract

“…The yoke and reward-retrieval conditions showed large increases during both behavior phases. Accumbal dopamine release has been proposed to be generally required for associative learning and/or motivated or effortful behavior (Robbins et al, 1990;Garris et al, 1999;Smith-Roe and Kelley, 2000;Salamone and Correa, 2002;Goto and Grace, 2005). It is unlikely that motivational factors per se account for different dopamine profiles between the three behavior groups because there were no significant between-group differences in the time required to complete a trial, and all rats consumed all available food pellets.…”

Section: Discussionmentioning

confidence: 99%

Rule Learning and Reward Contingency Are Associated with Dissociable Patterns of Dopamine Activation in the Rat Prefrontal Cortex, Nucleus Accumbens, and Dorsal Striatum

Stefani¹,

Moghaddam²

2006

J. Neurosci.

123

115

View full text Add to dashboard Cite

The midbrain dopamine system has been ascribed roles in reward expectancy, error detection, prediction, and memory. However, these theories typically do not differentiate between dopamine response and action in different forebrain terminal fields. We measured dopamine release in the prefrontal cortex (PFC), nucleus accumbens (NAc), and dorsal striatum (DS) of rats exposed to the same maze apparatus under three behavioral conditions: a set-shift task in which reward depended on discrimination learning and extradimensional set-shifting, a yoked condition in which reward was intermittent and not under the control of the subject, and a "reward-retrieval" variant in which reward was certain on every trial. We found dissociable patterns of dopamine release associated with learning, uncertainty, and reward. Dopamine increased in all three regions when reward was contingent on rule learning and shifting or was uncertain. These increases were sustained after behavior. There was a significant correlation between the magnitude of increase in PFC dopamine and the rapidity with which rats shifted between discrimination rules. In the yoke condition, in which the receipt of reward was always uncertain, the opposite relationship between dopamine levels and likelihood of reward was observed. Predictable, noncontingent reward was associated with increased dopamine levels in the NAc and DS. In contrast, PFC dopamine did not increase significantly above baseline levels. Thus, the dopaminergic projections to the PFC and nucleus accumbens were selectively, yet differentially, activated in situations of uncertainty and cognitive demand, whereas the dopaminergic projection to the DS responded independently of task differences in learning and reward.

show abstract

Effects of dopamine depletion from the caudate-putamen and nucleus accumbens septi on the acquisition and performance of a conditional discrimination task

Cited by 132 publications

References 38 publications

PVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm.

PVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm.

Postnatal manganese exposure alters dopamine transporter function in adult rats: Potential impact on nonassociative and associative processes

Rule Learning and Reward Contingency Are Associated with Dissociable Patterns of Dopamine Activation in the Rat Prefrontal Cortex, Nucleus Accumbens, and Dorsal Striatum

Contact Info

Product

Resources

About