Dopamine and proximity in motivation and cognitive control

Westbrook, Andrew; Frank, Michael J.

doi:10.1016/j.cobeha.2017.12.011

Cited by 50 publications

(45 citation statements)

References 72 publications

(94 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is consistent with previous findings that extended training makes learned responses habitual and less flexible [54,55]. Recent studies have found that dopamine activity is closely linked to initiating a movement or a sequence of learned actions [8,9,19,23,24,52,[56][57][58]. These findings, along with our simulation, suggest that during early training when an animal is uncertain about the key stimuli for the given task, many effective task dimensions-each generating a weak prediction error signal and affecting animal's behavior for a short while-guide the animal to complete the task.…”

Section: The Transition From Phasic To Ramping Patterns Upon Changes supporting

confidence: 92%

“…Recent empirical studies have found an increasingly diverse repertoire of dopamine activity that cannot be interpreted as a prediction error signal [17,[58][59][60][61][62][63][67][68][69][70]. Our hypothesis examining the role of dopamine in resource allocation provides a framework to understand the diverse dopamine activities; prediction error is a major (but not the only) factor that determines resource allocation.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Ramping and phasic dopamine activity accounts for efficient cognitive resource allocation during reinforcement learning

Song

2018

Preprint

View full text Add to dashboard Cite

Dopamine activity may transition between two patterns: phasic responses to reward-predicting cues and ramping activity arising when an agent approaches the reward. However, when and why dopamine activity transitions between these modes is not understood. We hypothesize that the transition between ramping and phasic patterns reflects resource allocation which addresses the task dimensionality problem during reinforcement learning (RL). By parsimoniously modifying a standard temporal difference (TD) learning model to accommodate a mixed presentation of both experimental and environmental stimuli, we simulated dopamine transitions and compared it with experimental data from four different studies. The results suggested that dopamine transitions from ramping to phasic patterns as the agent narrows down candidate stimuli for the task; the opposite occurs when the agent needs to re-learn candidate stimuli due to a value change. These results lend insight into how dopamine deals with the tradeoff between cognitive resource and task dimensionality during RL.

show abstract

Section: The Transition From Phasic To Ramping Patterns Upon Changes supporting

confidence: 92%

Section: Discussionmentioning

confidence: 99%

Ramping and phasic dopamine activity accounts for efficient cognitive resource allocation during reinforcement learning

Song

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Much of the previous research in this area has used bandittype designs to better understand dopaminergic functions, which although useful, may not always reflect real-world problems. Furthermore, in such experiments, foreground and background reward rates can become correlated, such that the value of exploring alternatives has an instrumentally predictive value of obtaining an immediate (foreground) reward Kayser et al, 2015;Westbrook and Frank, 2018). However, in ecological settings, choices to "leave" a patch and explore are not choices between two stimuli with a predictive value but instead involve traveling to obtain rewards elsewhere.…”

Section: Discussionmentioning

confidence: 99%

Dopamine Modulates Dynamic Decision-Making during Foraging

et al. 2020

View full text Add to dashboard Cite

The mesolimbic dopaminergic system exerts a crucial influence on incentive processing. However, the contribution of dopamine in dynamic, ecological situations where reward rates vary, and decisions evolve over time, remains unclear. In such circumstances, current (foreground) reward accrual needs to be compared continuously with potential rewards that could be obtained by traveling elsewhere (background reward rate), to determine the opportunity cost of staying versus leaving. We hypothesized that dopamine specifically modulates the influence of background, but not foreground, reward information when making a dynamic comparison of these variables for optimal behavior. On a novel foraging task based on an ecological account of animal behavior (marginal value theorem), human participants of either sex decided when to leave locations in situations where foreground rewards depleted at different rates, either in rich or poor environments with high or low background reward rates. In line with theoretical accounts, people's decisions to move from current locations were independently modulated by changes in both foreground and background reward rates. Pharmacological manipulation of dopamine D2 receptor activity using the agonist cabergoline significantly affected decisions to move on, specifically modulating the effect of background reward rates. In particular, when on cabergoline, people left patches in poor environments much earlier. These results demonstrate a role of dopamine in signaling the opportunity cost of rewards, not value per se. Using this ecologically derived framework, we uncover a specific mechanism by which D2 dopamine receptor activity modulates decision-making when foreground and background reward rates are dynamically compared.

show abstract

“…In a randomized, repeated-measures design, we also administered placebo, 45 methylphenidate (a dopamine and noradrenaline reuptake blocker), and sulpiride (a selective D2 receptor agent) while participants made explicit cost-benefit decisions about whether to engage in cognitive effort. We further monitored participants' gaze to cost and benefit attributes and assessed how fixations interacted with attribute values and dopamine to impact dynamic decision processes (8,(22)(23)(24). 50…”

mentioning

confidence: 99%

Dopamine Promotes Cognitive Effort by Biasing the Benefits Versus Costs of Cognitive Work

Westbrook

Bosch

Määttä

et al. 2019

Preprint

View full text Add to dashboard Cite

Stimulants like methylphenidate are increasingly used for cognitive enhancement, but precise mechanisms are unknown. Here we show that methylphenidate boosts willingness to expend cognitive effort by altering the benefit-to-cost ratio of cognitive work. Willingness to 15 expend effort was greater for participants with higher striatal dopamine synthesis capacity, while methylphenidate and sulpiride -a selective D2 receptor agent -increased cognitive motivation more for participants with lower synthesis capacity. A sequential sampling model informed by momentary gaze revealed that decisions to expend effort are related to amplification of benefitversus-cost information attended early in the decision process, while the effect of benefits is 20 strengthened with higher synthesis capacity and by methylphenidate. These findings demonstrate that methylphenidate boosts the perceived benefits-versus-costs of cognitive effort by modulating striatal dopamine signaling. Main Text:Cognitive control is subjectively effortful, causing people to avoid demanding tasks (1) 25 and to discount goals (2, 3), but incentives can offset these costs (2). Striatal dopamine invigorates physical action by mediating the tradeoff between physical costs and benefits (4). In cortico-striatal loops governing action selection, dopamine has opponent effects on D1 and D2expressing medium spiny neurons, which are thought to modulate the relative sensitivity to the benefits versus the costs of actions (5). Given that similar mechanisms are thought to govern 30 cognitive action selection (6-8), we hypothesized that striatal dopamine could promote willingness to exert cognitive effort, boosting motivated cognitive control for attention, planning, and decision-making (9-12).Converging evidence from research on Parkinson's disease (13)(14)(15)(16)(17), showing dopaminedependent changes in cognitive motivation, provides an initial basis for this conjecture. 35 Moreover, catecholamine-enhancing psychostimulants alter cognitive effort-based choice in both rodents (10) and humans (18). This raises the question of whether commonly used "smart drugs" act by enhancing the willingness rather than ability to exert cognitive control. Indeed, the dominant interpretation of stimulant effects is that they improve cognitive processing, via direct effects on cortical areas,

show abstract

Dopamine and proximity in motivation and cognitive control

Cited by 50 publications

References 72 publications

Ramping and phasic dopamine activity accounts for efficient cognitive resource allocation during reinforcement learning

Ramping and phasic dopamine activity accounts for efficient cognitive resource allocation during reinforcement learning

Dopamine Modulates Dynamic Decision-Making during Foraging

Dopamine Promotes Cognitive Effort by Biasing the Benefits Versus Costs of Cognitive Work

Contact Info

Product

Resources

About