Dopaminergic modulation of human inter-temporal choice: a diffusion model analysis using the D2-receptor-antagonist haloperidol

Abstract

14The neurotransmitter dopamine is implicated in diverse functions, including reward 15 processing, reinforcement learning and cognitive control. The tendency to discount future 16 rewards in value over time has long been discussed in the context of potential dopaminergic 17 modulation. Here we examined the effect of a single dose of the D2 receptor antagonist 18 Haloperidol (2mg) on temporal discounting. Our approach extends previous human 19 pharmacological studies in two ways. First, we applied state-of-the-art computational 20 modeling based on the drift diffusion model to comprehensively examine choice dynamics. 21Second, we examined dopaminergic modulation of reward magnitude effects on temporal 22 discounting. Drift diffusion modeling revealed reduced temporal discounting and substantially 23 faster non-decision times under Haloperidol. Temporal discounting was substantially 24 increased for low vs. high reward magnitudes, but this magnitude effect was largely 25 unaffected by Haloperidol. These results were corroborated by model-free analyses as well as 26 modeling via more standard approaches using softmax action selection. We previously 27 reported elevated caudate activation under Haloperidol in this sample of participants, 28 supporting the idea that Haloperidol elevated dopamine neurotransmission, e.g. by blocking 29 inhibitory feedback via presynaptic D2 autoreceptors. The present modeling results show that 30 during inter-temporal choice, this leads to attenuated temporal discounting and increased 31 response vigor (shorter non-decision times).32 33planning is thought to rely on anticipatory mechanisms that compute future outcomes based 68 on an environmental state-space, whereas such explicit forward planning is not required for 69 model-free RL. While one study found no association between model-based RL and temporal 70 discounting 19 , another large-scale online study observed the predicted positive association: 71 participants with a stronger contribution of model-based planning to RL also showed reduced 72 temporal discounting 20 . However, in contrast to temporal discounting, where (as mentioned 73 above) elevation of DA levels via the catecholamine precursor L-DOPA lead to more 74 impulsive decision-making and steeper discounting 13 (though results in a larger sample were 75 considerably more mixed 14 ), L-DOPA instead increased reliance on model-based RL in 76 controls 21 and Parkinson's disease patients 22 . However, a recent study in a substantially larger 77 sample (n=65) did not replicate this overall effect of L-DOPA on model-based control 23 . 78Rather, in this study the effect of L-DOPA was restricted to a subset of participants with high 79 working memory capacity. 80Several task-related and contextual factors affect the degree of temporal 81 discounting 1,24 . One well-replicated behavioral effect, the magnitude effect, refers to the 82 observation that the rate of temporal discounting decreases with increasing reward 83 magnitude 25 . In humans, this effect depends on lateral p...

Breaking Deadlocks: Reward Probability and Spontaneous Preference Shape Voluntary Decisions and Electrophysiological Signals in Humans

“…In traditional value-based decisions assumed by the prospect theory, a decision-maker needs to integrate the value and probability of gain or loss to obtain an expected utility for each option (Tversky and Kahneman 1992). Together, our results here and previous studies (Wagner et al 2020) provide converging evidence that both reward value and probability can influence RT in equal choices. This raises the intriguing possibility of our results to be generalized to choices with the same expected utility but the different combinatory of value and probability.…”

“…Furthermore, the accumulation rate is associated with the individual differences in working memory (Schmiedek et al 2007) and attention (Nunez et al 2017), while the non-decision time is faster in individuals with higher diffusion MRI-derived neurite density in the corticospinal tract, the primary motor output pathway (Karahan et al 2019). Recent research showed that both parameters can be influenced by reward magnitude (Wagner et al 2020), and the current study will examine further whether reward probability and preference influence the two model parameters.…”

Reliability assessment of temporal discounting measures in virtual reality environments

“…First, leveraging the information contained in the full RT distributions can improve the stability of parameter estimates [37,38] . Second, by conceiving decision making as a dynamic diffusion process, a more detailed picture of the underlying latent processes emerges [39][40][41][42][43] . Recent studies, for instance, applied these techniques to temporal discounting, where they revealed novel insights into effects of pharmacological manipulation of the dopamine system on choice dynamics [42] .…”

“…Next, we incorporated response times (RTs) into the model by replacing the softmax choice rule with the drift diffusion model (DDM) [39][40][41][42] . The DDM models choices between two options as a noisy evidence accumulation that terminates as soon as the accumulated evidence exceeds one of two boundaries.…”

“…One drawback of a linear representation of the relationship between the drift-rate v and trial-wise value differences is that v might increase infinitely with high value differences, which can lead the model to under-predict RTs for high value differences [41] . In line with previous work [41,42] we thus included a third version of the DDM, that assumes a non-linear sigmoidal mapping from trial-wise value differences to drift rates (eq. 5 and 6) [39] :…”