Decomposition of reinforcement learning deficits in gambling disorder via drift diffusion modeling and functional magnetic resonance imaging

Abstract

Gambling disorder is associated with deficits in classical feedback-based learning tasks, but the computational mechanisms underlying such learning impairments are still poorly understood. Here, we examined this question using a combination of computational modeling and functional resonance imaging (fMRI) in gambling disorder participants (n=23) and matched controls (n=19). Participants performed a simple reinforcement learning task with two pairs of stimuli (80% vs. 20% reinforcement rates per pair). As predicted, gamblers made significantly fewer selections of the optimal stimulus, while overall response times (RTs) were not significantly different between groups. We then used comprehensive modeling using reinforcement learning drift diffusion models (RLDDMs) in combination with hierarchical Bayesian parameter estimation to shed light on the computational underpinnings of this performance impairment. In both groups, an RLDDM in which both non-decision time and response threshold (boundary separation) changed over the course of the experiment accounted for the data best. The model showed good parameter recovery, and posterior predictive checks revealed that in both groups, the model reproduced the evolution of both accuracy and RTs over time. Examination of the group-wise posterior distributions revealed that the learning impairment in gamblers was attributable to both reduced learning rates and a more rapid reduction in boundary separation over time, compared to controls. Furthermore, gamblers also showed substantially shorter non-decision times. Model-based imaging analyses then revealed that value representations in gamblers in the ventromedial prefrontal cortex were attenuated compared to controls, and these effects were partly associated with model-based learning rates. Exploratory analyses revealed that a more anterior ventromedial prefrontal cortex cluster showed attenuations in value representations in proportion to gambling disorder severity in gamblers. Taken together, our findings reveal computational mechanisms underlying reinforcement learning impairments in gambling disorder, and confirm the ventromedial prefrontal cortex and as a critical neural hub in this disorder.

Reliability assessment of temporal discounting measures in virtual reality environments

“…Model comparison revealed that the DDM S had the lowest DIC in all conditions (Table 4) replicating previous work [45,46,48] . Consequently, further analyses of session effects and reliability focused on this model.…”

“…This was observed for model-free measures (AUC), as well as the log(k) parameter of the hyperbolic discounting model with the softmax choice rule and the drift diffusion model with non-linear drift rate scaling (DDM S ). Model comparison revealed that the DDM S accounted for the data best, confirming previous findings [43,45,46,48] . Although generally, discount rates assessed in the three sessions were of similar magnitude, in the DDM S there was moderate evidence for reduced discounting (i.e., smaller values of log(k) ) in the VR neutral session.…”

“…Finally, the good test-retest reliability of log(k) from the DDMs furthermore indicates that RTs obtained in VR can meaningfully be modeled using the DDM. The potential utility of this modeling approach in the context of gambling disorder is illustrated by a recent study that reported reduced boundary separation ( ) in pathological gamblers compared to healthy controls in a reinforcement learning task [44] . Given that there are mixed results when it comes to the effect of addiction related cues on RTs [74][75][76] , the effects of these cues on the latent decision variables included in the DDM could provide additional insights.…”

“…Likewise, we applied these techniques to examine the processes underlying reinforcement learning impairments in gambling disorder [44] and decision-making alterations following medial orbitofrontal cortex lesions [41] . Importantly, most standard lab-based testing settings use keyboards, button boxes and computer screens to record responses and display stimuli during behavioral tasks.…”

“…Recent studies, for instance, applied these techniques to temporal discounting, where they revealed novel insights into effects of pharmacological manipulation of the dopamine system on choice dynamics [46] . Likewise, we applied these techniques to examine the processes underlying reinforcement learning impairments in gambling disorder [48] and decision-making alterations following medial orbitofrontal cortex lesions [45] . Importantly, most standard lab-based testing settings use keyboards, button boxes and computer screens to record responses and display stimuli during behavioral tasks.…”