Value generalization in human avoidance learning

Abstract

SummaryGeneralization during aversive decision-making allows us to avoid a broad range of potential threats following experience with a limited set of exemplars. However, over-generalization, resulting in excessive and inappropriate avoidance, has been implicated in a variety of psychological disorders. Here, we use reinforcement learning modelling to dissect out different contributions to the generalization of instrumental avoidance in two groups of human volunteers (N=26, N=482). We found that generalization of avoidance could be parsed into perceptual and value-based processes, and further, that value-based generalization could be subdivided into that relating to aversive and neutral feedback - with corresponding circuits including primary sensory cortex, anterior insula, and ventromedial prefrontal cortex, respectively. Further, generalization from aversive, but not neutral, feedback was associated with self-reported anxiety and intrusive thoughts. These results reveal a set of distinct mechanisms that mediate generalization in avoidance learning, and show how specific individual differences within them can yield anxiety.

Associations between aversive learning processes and transdiagnostic psychiatric symptoms in a general population sample

“…Overall, the present results add to the growing literature showing associations between psychopathology and learning under uncertainty. Previous studies using computational approaches have largely focused on learning about rewards and losses [10][11][12]27,42 , or perceptual learning 9 , and those that have used more aversive paradigms (using outcomes intended to evoke subjective anxiety), such as learning to predict electric shocks, have been limited by small samples 5,15,18,43 . While there is a rich literature using simple fear conditioning paradigms to investigate aversive learning in individuals with anxiety disorders 44,45 , these tasks typically do not manipulate uncertainty, as was the intention in our task.…”

“…However, it has been difficult to examine aversive learning in online environments, as aversive lab stimuli such as shock cannot be easily administered online. Only one study thus far has investigated threat-related decision-making (although not learning) online, using monetary loss as an aversive stimulus 27 . A game-based design allowed us to design a task that required avoidance behaviour as well as evoke feelings of anxiety, taking advantage of the well-known ability of games to produce strong emotional reactions [47][48][49][50][51] .…”

“…However, these approaches have not been designed optimally for identifying mechanisms that span traditional diagnostic boundaries. This assumes importance in light of recent studies, using large samples, showing several aspects of learning and decision-making relate more strongly to transdiagnostic factors (symptom dimensions that are not unique to any one disorder) than to any specific categorical conception of psychiatric disorder 6,8,[25][26][27] . Applying such an approach to aversive learning may yield better insights into the role of learning in psychiatric disorders.…”

Prolonged avoidance training exacerbates OCD-like behaviors in a rodent model

“…In the context of OCD, compulsions often represent anxious avoidance responses. Therefore, each time a compulsion is performed, it might relieve anxiety which would further reinforce avoidance (negative reinforcement) [49][50][51] . This might resemble the prolonged conditioning in our 20d rats.…”

Using reinforcement learning models in social neuroscience: frameworks, pitfalls, and suggestions of best practices

“…In its simplest form, the Rescorla-Wagner model, observations are explained by Pavlovian conditioning (Rescorla and Wagner, 1972). It has been applied to, for example, category learning (Poldrack and Foerde, 2008; Ashby and Maddox, 2011), Pavlovian and instrumental learning in reward and punishment conditions (Daw et al, 2006;Dolan and Dayan, 2013;Swart et al, 2017), as well as fear conditioning (Koizumi et al, 2017;Lindström et al, 2018;Norbury et al, 2018).…”