Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning

Abstract

BackgroundSerious and debilitating symptoms of anxiety are the most common mental health problem worldwide, accounting for around 5% of all adult years lived with disability in the developed world. Avoidance behavior—avoiding social situations for fear of embarrassment, for instance—is a core feature of such anxiety. However, as for many other psychiatric symptoms the biological mechanisms underlying avoidance remain unclear.MethodsReinforcement learning models provide formal and testable characterizations of the mechanisms of decision making; here, we examine avoidance in these terms. A total of 101 healthy participants and individuals with mood and anxiety disorders completed an approach-avoidance go/no-go task under stress induced by threat of unpredictable shock.ResultsWe show an increased reliance in the mood and anxiety group on a parameter of our reinforcement learning model that characterizes a prepotent (pavlovian) bias to withhold responding in the face of negative outcomes. This was particularly the case when the mood and anxiety group was under stress.ConclusionsThis formal description of avoidance within the reinforcement learning framework provides a new means of linking clinical symptoms with biophysically plausible models of neural circuitry and, as such, takes us closer to a mechanistic understanding of mood and anxiety disorders.

Stimulation of the vagus nerve reduces learning in a go/no-go reinforcement learning task

“…Furthermore, reported anti-depressant effects of tVNS may partly rely on reduced learning, especially from punishments, as this may compensate for the reported increased punishment sensitivity in depressed patients [46,69]. More broadly, reduced dependence on learned contingencies may also offer the possibility to prevent overreliance on learned action-outcome combinations and encourage exploration.…”

“…Additionally, participant's choices in the tVNS condition were 'noisier' and less dependent on learned action values (Δξ = 0.035, p = .086, pboot = .050), although only nominally significant before correction for multiple testing. We also explored more complex models by additionally separating reward sensitivity and/or Pavlovian bias for reward and punishment as previously described [46]. However, these models did not provide a more parsimonious account at the individual level compared to the simpler models and individual estimates became increasingly unstable across iterations precluding their use to reliably estimate withinsubject stimulation effects (see SI).…”

A dimensional approach to determine common and specific neurofunctional markers for depression and social anxiety during emotional face processing

“…With regard to social anxiety only the pathway between the putamen and middle cingulate cortex demonstrated increased connectivity. Both regions share functional connections (Hoffstaedter et al, 2012), thought to underlie the integration of emotion and motor-responses (Shackman et al, 2011), with stronger connectivity in this pathway during negative face possibly reflecting increased tendency to avoid or withdrawal negative social signals which constitutes a core feature of social anxiety disorders (Mkrtchian, Aylward, Dayan, Roiser, & Robinson, 2017).…”

Social reinforcement learning as a predictor of real-life experiences in individuals with high and low depressive symptomatology

“…Fitting these models to participants' choice behavior by adjusting model parameters allows for the assessment of group differences in various aspects of the learning process. Using this approach, it has been found that depressed participants are less sensitive to rewarding outcomes (Huys et al, 2013) but more responsive to punishments (Byrne et al, 2016;Mkrtchian et al, 2017) than controls. Additionally, depression has been associated with abnormalities in learning rate (Chase et al, 2010;Dombrovski et al, 2010;Beevers et al, 2013;Cooper et al, 2014) and exploration (Kunisato et al, 2012;Beevers et al, 2013;Rupprechter et al, 2018) parameters, with results suggesting that depressed individuals show suboptimal reward learning parameters.…”