Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism

Abstract

Abstract& Dopamine (DA) depletion in the basal ganglia (BG) of Parkinson's patients gives rise to both frontal-like and implicit learning impairments. Dopaminergic medication alleviates some cognitive deficits but impairs those that depend on intact areas of the BG, apparently due to DA ''overdose.'' These findings are difficult to accommodate with verbal theories of BG/DA function, owing to complexity of system dynamics: DA dynamically modulates function in the BG, which is itself a modulatory system. This article presents a neural network model that instantiates key biological properties and provides insight into the underlying role of DA in the BG during learning and execution of cognitive tasks. Specifically, the BG modulates the execution of ''actions'' (e.g., motor responses and working memory updating) being considered in different parts of the frontal cortex. Phasic changes in DA, which occur during error feedback, dynamically modulate the BG threshold for facilitating/suppressing a cortical command in response to particular stimuli. Reduced dynamic range of DA explains Parkinson and DA overdose deficits with a single underlying dysfunction, despite overall differences in raw DA levels. Simulated Parkinsonism and medication effects provide a theoretical basis for behavioral data in probabilistic classification and reversal tasks. The model also provides novel testable predictions for neuropsychological and pharmacological studies, and motivates further investigation of BG/DA interactions with the prefrontal cortex in working memory. &

Anatomy of a decision: Striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal.

“…This is consistent with observations that DA enhances synaptic plasticity and promotes long-term potentiation via D 1 receptors (go pathway) while promoting long-term depression via D 2 receptors (no-go; Centonze, Picconi, Gubellini, Bernardi, & Calabresi, 2001;Nishi, Snyder, & Greengard, 1997). Conversely, DA dips during negative reinforcement may also be adaptive, in that they can drive no-go learning to avoid selecting the nonreinforced response in the future (Frank, 2005). 1 Specifically, low 1 Although the change in firing rate associated with DA dips is smaller than that of the bursts (due to already low baseline firing rates of DA cells; Bayer & Glimcher, 2005), this asymmetry does not mean that dips are less effective in driving learning.…”

Reduced hedonic capacity in major depressive disorder: Evidence from a probabilistic reward task

“…Moreover, it is possible that blunted response bias in MDD subjects might be partially explained by an impairment in learning that the lean stimulus is not associated with frequent reward (cf. Frank, 2005). Although future studies will be required for conclusive tests of these alternative interpretations, a convergence of findings points to blunted reward responsiveness in depression.…”

Opponent actor learning (OpAL): Modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive.

“…We aim to provide a theoretically simple algorithmic model, with parameters and variables that can easily be related to biologically interpretable measures of interest, such as tonic or phasic dopamine level, D1 and D2-expressing striatal neuronal activity or synaptic strengths, synaptic plasticity etc. Our approach is inspired by two distinct levels of modeling: on one hand, the well-known and widely used actor-critic algorithm (Sutton & Barto, 1998); on the other hand, the more biologically detailed neural network description of corticobasal ganglia loops including multiple pathways (Frank, 2005). Although previous attempts to link these levels of modeling exist, these did not consider separate valuation systems for action selection and learning, or the effects of incentive but, rather, included a single value for each action and only allowed for asymmetric learning rates for positive versus negative prediction errors (Doll, Hutchison, & Frank, 2011;Frank, Moustafa, et al, 2007).…”

“…Thus overall, increases in dopamine act to preferentially emphasize processing in the D1 facilitatory pathway and to suppress processing in the D2 suppressive pathway, whereas decrease in dopamine have the opposite effect, potentiating the D2 pathway. This has been proposed as the mechanism by which DA promotes approach learning in the direct pathway and avoidance learning in the indirect pathway (Frank, 2005), with opposite-coding but apparently redundant representations of action values and learning. Although the direct (D1-MSNs) and indirect (D2-MSNs) pathways are often labeled as Go and NoGo pathways due to their link to approach and avoidance, respectively, in the models they do not just encode a message signaling to go or not, but rather the aggregated evidence in favor of each action versus against that action.…”