Testing Computational Models of Dopamine and Noradrenaline Dysfunction in Attention Deficit/Hyperactivity Disorder

Abstract

We test our neurocomputational model of fronto-striatal dopamine (DA) and noradrenaline (NA) function for understanding cognitive and motivational deficits in attention deficit/hyperactivity disorder (ADHD). Our model predicts that low striatal DA levels in ADHD should lead to deficits in 'Go' learning from positive reinforcement, which should be alleviated by stimulant medications, as observed with DA manipulations in other populations. Indeed, while nonmedicated adult ADHD participants were impaired at both positive (Go) and negative (NoGo) reinforcement learning, only the former deficits were ameliorated by medication. We also found evidence for our model's extension of the same striatal DA mechanisms to working memory, via interactions with prefrontal cortex. In a modified AX-continuous performance task, ADHD participants showed reduced sensitivity to working memory contextual information, despite no global performance deficits, and were more susceptible to the influence of distractor stimuli presented during the delay. These effects were reversed with stimulant medications. Moreover, the tendency for medications to improve Go relative to NoGo reinforcement learning was predictive of their improvement in working memory in distracting conditions, suggestive of common DA mechanisms and supporting a unified account of DA function in ADHD. However, other ADHD effects such as erratic trial-to-trial switching and reaction time variability are not accounted for by model DA mechanisms, and are instead consistent with cortical noradrenergic dysfunction and associated computational models. Accordingly, putative NA deficits were correlated with each other and independent of putative DA-related deficits. Taken together, our results demonstrate the usefulness of computational approaches for understanding cognitive deficits in ADHD.

Deficient reinforcement learning in medial frontal cortex as a model of dopamine-related motivational deficits in ADHD

“…Figure 2b shows the percentage of switches between options during the learning phase. Consistent with experimental data (Frank, et al, 2007a;Luman, et al, 2009) the ADHD group switched more often (t(19)=2.72, p=0.013), as a consequence of difficulty in learning the reward contingencies. This result cannot be attributed to differences in the action-selection process itself (SOFTMAX), because in both groups the action selection system had the same parameters.…”

“…The supporting evidence implicating putative RL-related brain networks is growing (Luman, Tripp, & Scheres, 2010), though behavioral evidence from classical RL schedules is currently limited (Sonuga-Barke, 2011). Nonetheless, several studies have documented impaired performance in ADHD patients during RL tasks, like probability tracking (Frank, Santamaria, O'Reilly, & Willcutt, 2007a;Luman, et al, 2009) or reward temporal discounting (Scheres, Tontsch, Thoeny, & Kaczkurkin, 2010). Indeed, although the literature lacks complete consistency (Scheres, et al, 2006), ADHD patients typically prefer immediate small over delayed large rewards, showing a steeper temporal discount curve (Marco, et al, 2009;Sagvolden, Aase, Zeiner, & Berger, 1998;Scheres, Lee, & Sumiya, 2008;Scheres, et al, 2010).…”

“…ADHD subjects have difficulties discovering the best option based on environmental outcomes (Frank, et al, 2007a;Luman, et al, 2009). Further, during learning, ADHD patients switch between options more than controls (Frank, et al, 2007a;Luman, et al, 2009).…”

“…Further, during learning, ADHD patients switch between options more than controls (Frank, et al, 2007a;Luman, et al, 2009). Here we administered a probabilistic choice task, in which two possible options were rewarded with different probabilities (75% and 25%, see Appendix), to both our ADHD (RVPM with reduced dopaminergic signal) and control model (intact RVPM).…”

Opponent actor learning (OpAL): Modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive.

“…As an example, nonmedicated Parkinson's patients have naturally low dopamine levels and exhibit better learning from negative than positive reward prediction errors, whereas the same patients while taking dopaminergic medication show better learning and choice based on positive outcomes but worse performance in avoiding negative outcomes (Bódi et al, 2009;Cools et al, 2009;Frank, Moustafa, et al, 2007;Frank, Seeberger, & O'Reilly, 2004;Moustafa, Sherman, & Frank, 2008;Palminteri, Boraud, Lafargue, Dubois, & Pessiglione, 2009;Smittenaar et al, 2012). Similar effects of dopamine manipulations have been observed in healthy and other populations (Cools et al, 2009;Frank, Moustafa, et al, 2007;Frank, Santamaria, Reilly, & Willcutt, 2007;Jocham et al, 2011;Pessiglione et al, 2006). This reinforcement learning theory of dopamine function can also account for other counterintuitive phenomena, such as aberrant learning in some situations (e.g., Beeler, Daw, Frazier, & Zhuang, 2010; Wiecki, Riedinger, von Ameln-Mayerhofer, Schmidt, & Frank, 2009: learned catalepsy), and provides a mechanism explaining progression of Parkinson's disease symptoms even without further dopaminergic degeneration (Beeler et al, 2012).…”