The drift diffusion model as the choice rule in inter-temporal and risky choice: a case study in medial orbitofrontal cortex lesion patients and controls

Abstract

<jats:title>Abstract</jats:title><jats:p>Sequential sampling models such as the drift diffusion model have a long tradition in research on perceptual decision-making, but mounting evidence suggests that these models can account for response time distributions that arise during reinforcement learning and value-based decision-making. Building on this previous work, we implemented the drift diffusion model as the choice rule in inter-temporal choice (<jats:italic>temporal discounting</jats:italic>) and risky choice (<jats:italic>probability discounting</jats:italic>) using a hierarchical Bayesian estimation scheme. We validated our approach in data from nine patients with focal lesions to the ventromedial prefrontal cortex / medial orbitofrontal cortex (vmPFC/mOFC) and nineteen age- and education-matched controls. Choice model parameters estimated via standard softmax action selection were reliably reproduced using the drift diffusion model as the choice rule, both for temporal discounting and risky choice. Model comparison revealed that, for both tasks, the data were best accounted for by a variant of the drift diffusion model including a non-linear mapping from value-differences to trial-wise drift rates. Posterior predictive checks of the winning models revealed a reasonably good fit to individual participants reaction time distributions. We then applied this modeling framework and 1) reproduced our previous results regarding temporal discounting in vmPFC/mOFC patients and 2) showed in a previously unpublished data set on risky choice that vmPFC/mOFC patients exhibit increased risk-taking relative to controls. Analyses of diffusion model parameters revealed that vmPFC/mOFC damage abolished neither value sensitivity nor asymptote of the drift rate. Rather, it substantially increased non-decision times and reduced response caution during risky choice. Our results highlight that novel insights can be gained from applying sequential sampling models in studies of inter-temporal and risky decision-making in cognitive neuroscience.</jats:p>

Dopaminergic modulation of human inter-temporal choice: a diffusion model analysis using the D2-receptor-antagonist haloperidol

“…The null model (DDM 0 ) assumes fixed drift rates independent of value. We compared this model to two variants of the DDM that assume that trial-wise drift rates depend on the value-differences between options, either in a linear fashion (DDM lin ) 32 or in a non-linear (sigmoid) fashion 30,38 . The data were best accounted for by the DDM with non-linear value-scaling of trial-wise drift rates (Table 2).…”

“…We applied a recent class of computational models combining standard reinforcement learning and/or valuation models with the drift diffusion model 30,32,33,38 . Such a comprehensive analysis of RTs was not possible in most previous studies due to the specifics of task timing 13 or the available trial numbers 14 that precluded RT-based modeling 14,16 .…”

“…Note that for each parameter x , we also include a parameter s x that models the change in that parameter from the high magnitude (SS=100) to the low magnitude (SS=20) condition (coded via a the dummy-coded condition regressor I t ). As in previous work 30,32,38 , we then set up temporal discounting diffusion models by making trial-wise drift rates proportional to the difference in subjective values between options. First, we set up a linear modeling scheme (DDM lin ) 32 : …”

“…As noted above, RTs for SS options were multiplied by −1 prior to model estimation, such that this formulation predicts SS choices whenever SV(SS)>SV(LL) (the trial-wise drift rate is negative), and predicts longest RTs for trials with the highest decision-conflict (i.e., in the case of SV(SS)= SV(LL) the trial-wise drift rate is zero). We next examined a DDM with non-linear trial-wise drift rate scaling (DDM S ) that has recently been reported to account for the value-dependency of RTs better than the DDM lin 30,38 . In this model, the scaled value difference from Eq.…”

“…This observation is compatible with a predominantly presynaptic effect of Haloperidol in the present study, leading to an overall increase in striatal dopaminergic signaling. Importantly, we extend previous pharmacological studies by applying a modeling framework for temporal discounting data that is based on a combination of standard discounting models with the drift diffusion model (DDM) 30–33 , allowing us to comprehensively examine drug effects on RT components related to both valuation and non-valuation related processes.…”