Model-Based Control in Dimensional Psychiatry

Voon, Valerie; Reiter, Andrea; Sebold, Miriam; Groman, Stephanie M.

doi:10.1016/j.biopsych.2017.04.006

Cited by 101 publications

(116 citation statements)

References 76 publications

Supporting

Mentioning

105

Contrasting

Order By: Relevance

“…Two distinct systems influence our choice behavior: goal directed and habitual control. Goal-directed (or model-based) control is characterized by a learned internal model of the environment that can dynamically evaluate optimal actions, a flexible but computationally expensive strategy [1][2][3] . By contrast, habitual (or modelfree) control computes the value of each action entirely by past experience (reward prediction errors), sacrificing flexibility for greater efficiency.…”

Section: Introductionmentioning

confidence: 99%

“…By contrast, habitual (or modelfree) control computes the value of each action entirely by past experience (reward prediction errors), sacrificing flexibility for greater efficiency. Disruptions in the balance of these strategies may underlie a range of pathological behaviours, in particular psychiatric disorders characterized by compulsivity [3][4][5] .…”

Section: Introductionmentioning

confidence: 99%

“…This balance between goal-directed and habitual strategies is mediated by various neurochemical processes. Among these, the dopamine system is most frequently implicated; a smaller number of studies also point to the involvement of the serotonin and opioid systems 3,6 . The role of dopamine in this balance is a topic of some debate.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The neurochemical substrates of habitual and goal-directed control

et al. 2020

Self Cite

View full text Add to dashboard Cite

Our daily decisions are governed by the arbitration between goal-directed and habitual strategies. However, the neurochemical basis of this arbitration is unclear. We assessed the contribution of dopaminergic, serotonergic, and opioidergic systems to this balance across reward and loss domains. Thirty-nine participants (17 healthy controls, 15 patients with pathological gambling, and 7 with binge eating disorder) underwent positron emission tomography (PET) imaging with [ 18 F]FDOPA, [ 11 C]MADAM and [ 11 C]carfentanil to assess presynaptic dopamine, and serotonin transporter and mu-opioid receptor binding potential. Separately, participants completed a modified two-step task, which quantifies the degree to which decision-making is influenced by goal-directed or habitual strategies. All participants completed a version with reward outcomes; healthy controls additionally completed a version with loss outcomes. In the context of rewarding outcomes, we found that greater serotonin transporter binding potential in prefrontal regions was associated with habitual control, while greater serotonin transporter binding potential in the putamen was marginally associated with goal-directed control; however, the findings were no longer significant when controlling for the opposing valence (loss). In blocks with loss outcomes, we found that the opioidergic system, specifically greater [ 11 C]carfentanil binding potential, was positively associated with goal-directed control and negatively associated with habit-directed control. Our findings illuminate the complex neurochemical basis of goaldirected and habitual behavior, implicating differential roles for prefrontal and subcortical serotonin in decisionmaking across healthy and pathological populations.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The neurochemical substrates of habitual and goal-directed control

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…The assertion that variation exists across individuals in the specific cortical-striatal networks that underpin the expression of appetitive behavior is supported by a rich literature including the well characterized spectrum of goal-directed to habitual behavior [41–44]. Thus, the striatal sub-regions driving binge like behavior could vary across individuals and impact which striatal target (NAc shell vs. core) is most likely to modulate binge behavior.…”

Section: Discussionmentioning

confidence: 99%

Machine learning based classification of deep brain stimulation outcomes in a rat model of binge eating using ventral striatal oscillations

Doucette

Dwiel

Boyce

et al. 2018

Preprint

View full text Add to dashboard Cite

Neuromodulation-based interventions continue to be evaluated across an array of appetitive disorders but broader implementation of these approaches remains limited due to variable treatment outcomes. We hypothesize that individual variation in treatment outcomes may be linked to differences in the networks underlying these disorders. Here, Sprague-Dawley rats received deep brain stimulation separately within each nucleus accumbens (NAc) sub-region (core and shell) using a within-animal crossover design in a rat model of binge eating.Significant reductions in binge size were observed with stimulation of either target but with significant variation in effectiveness across individuals. When features of local field potentials (LFPs) recorded from the NAc were used as predictors of the pre-defined stimulation outcomes (response or non-response) from each rat using a machine-learning approach (lasso), stimulation outcomes could be predicted with greater accuracy than expected by chance (effect sizes: core = 1.13, shell = 1.05). Further, these LFP features could be used to identify the best stimulation target for each animal (core vs. shell) with an effect size = 0.96. These data suggest that individual differences in underlying network activity may contribute to the variable outcomes of circuit based interventions and that measures of network activity have the potential to individually guide the selection of an optimal stimulation target and improve overall treatment response rates. peer-reviewed)

show abstract

“…These changes in turn aberrantly affect circuit gain and the stability of both ventral and dorsal cortico-striatal circuits, disrupting their respective role in encoding goal-directed behaviors (Balleine, 2005; Balleine and O’Doherty, 2010; Gruber and McDonald, 2012) and habitual responses (Yin et al, 2004; Balleine and O’Doherty, 2010). A similar effect is assumed for our algorithmic model, where over-evaluation of drugs and related RL affect the two control modalities, termed model-based and model-free , that loosely correspond to ventral/goal-oriented and dorsal/habitual implementations (Dolan and Dayan, 2013; Voon et al, 2017). As a result, and consistently with previous formulations of RL models of addiction (Redish et al, 2008; Piray et al, 2010; Gillan et al, 2016), both the planned evaluation of known action-outcome contingencies, represented in an internal model of the world, and the reactive immediate motor responses are biased towards drug-related selections.…”

Section: Introductionmentioning

confidence: 99%

A Multilevel Computational Characterization of Endophenotypes in Addiction

Fiore

Ognibene

Adinoff

et al. 2017

Preprint

View full text Add to dashboard Cite

Substance use disorders are characterized by a profound intersubject (phenotypic) variability in the expression of addictive symptomatology and propensity to relapse following treatment. However, laboratory investigations have primarily focused on common neural substrates in addiction, and have not yet been able to identify mechanisms that can account for the multifaceted phenotypic behaviors reported in literature. To investigate this knowledge gap theoretically, here we simulated phenotypic variations in addiction symptomology and responses to putative treatments, using both a neural model based on cortico-striatal circuit dynamics, and an algorithmic model of reinforcement learning. These simulations rely on the widely accepted assumption that both the ventral, model-based, goal-directed system and the dorsal, model-free, habitual system are vulnerable to extra-physiologic dopamine reinforcements triggered by drug consumption. We found that endophenotypic differences in the balance between the two circuit or control systems resulted in an inverted U-shape in optimal choice behavior. Specifically, greater unbalance led to a higher likelihood of developing addiction and more severe drug-taking behaviors. Furthermore, endophenotypes with opposite asymmetrical biases among cortico-striatal circuits expressed similar addiction behaviors, but responded differently to simulated treatments, suggesting personalized treatment development could rely on endophenotypic rather than phenotypic differentiations. We propose our simulated results, confirmed across neural and algorithmic levels of analysis, inform on a fundamental and, to date, neglected quantitative method to characterize clinical heterogeneity in addiction.

show abstract

Model-Based Control in Dimensional Psychiatry

Cited by 101 publications

References 76 publications

The neurochemical substrates of habitual and goal-directed control

The neurochemical substrates of habitual and goal-directed control

Machine learning based classification of deep brain stimulation outcomes in a rat model of binge eating using ventral striatal oscillations

A Multilevel Computational Characterization of Endophenotypes in Addiction

Contact Info

Product

Resources

About