Acute stress blunts prediction error signals in the dorsal striatum during reinforcement learning

Abstract

Reinforcement learning, which implicates learning from the rewarding and punishing outcomes of our choices, is critical for adjusted behaviour. Acute stress seems to affect this ability but the neural mechanisms by which it disrupts this type of learning are still poorly understood. Here, we investigate whether and how acute stress blunts neural signalling of prediction errors during reinforcement learning using model-based functional magnetic resonance imaging. Male participants completed a well-established reinforcement learning task involving monetary gains and losses whilst under stress and control conditions. Acute stress impaired participants' behavioural performance towards obtaining monetary gains, but not towards avoiding losses. Importantly, acute stress blunted signalling of prediction errors during gain and loss trials in the dorsal striatum -- with subsidiary analyses suggesting that acute stress preferentially blunted signalling of positive prediction errors. Our results thus reveal a neurocomputational mechanism by which acute stress may impair reward learning.

Acute stress alters probabilistic reversal learning in healthy male adults

“…However, in both studies, effectiveness of stressor was only confirmed with subjective ratings instead of physiological correlate such as cortisol or heart rate. Furthermore, the nature of stressors in both studies, namely, uncontrollable sounds (Carvalheiro et al, 2021) and threat of shock (Robinson et al, 2013), differs from our focus on psychosocial stress. Although both studies employed reward‐learning paradigms, they did not tap reversal learning specifically.…”

“…The few studies using within‐subjects designs to investigate learning are either purely behavioural (Radenbach et al, 2015) focused on psychoimmunological measures (Treadway et al, 2017) or neuroimaging methods such as electroencephalography (Cavanagh et al, 2011), which lacks the possibility of precise spatial signal localization and anatomical specificity with respect to the neural representation of RPE signals. To our knowledge, only two studies combine a within‐subjects design with computational modelling and functional neuroimaging (fMRI) to elucidate underlying cognitive mechanisms (Carvalheiro et al, 2021; Robinson et al, 2013). However, in both studies, effectiveness of stressor was only confirmed with subjective ratings instead of physiological correlate such as cortisol or heart rate.…”

Psychopathic traits and reinforcement learning under acute stress

“…The effects of acute stress on punishment learning are generally less well-understood (Porcelli & Delgado, 2017). Previous behavioral studies suggest that acute stress impairs reward learning, but the evidence for stress-induced deficits in punishment learning is scarcer (Berghorst et al, 2013;Carvalheiro et al, 2021aCarvalheiro et al, , 2021bPorcelli & Delgado, 2017). Reward learning has been robustly associated with dopaminergic functioning (Daw & Tobler, 2014;Glimcher, 2011;Pessiglione et al, 2006;Schultz et al, 1997), and it is known that acute stress increases dopamine release (Adler et al, 2000;Cabib & Puglisi-Allegra, 2012;Pruessner et al, 2004), which in turn might affect this type of learning.…”

“…Acute stress, albeit ubiquitous in everyday life, is known to impact reinforcement learning in healthy individuals (Porcelli & Delgado, 2017). Accumulating evidence suggests that acute stress impairs reward learning (Berghorst et al, 2013;Bogdan & Pizzagalli, 2006;Bogdan et al, 2011;Carvalheiro et al, 2021aCarvalheiro et al, , 2021bCremer et al, 2021;de Berker et al, 2016;Morris & Rottenberg, 2015;Paret & Bublatzky, 2020) and punishment learning (de Berker et al, 2016;Petzold et al, 2010), although the evidence for the latter is less robust (Aylward et al, 2019;Carvalheiro et al, 2021aCarvalheiro et al, , 2021bPorcelli & Delgado, 2017). Previous research indicates that individuals with high levels of psychopathic traits present blunted stress reactivity (Cima et al, 2008;Holi et al, 2006;House & Milligan, 1976;Johnson et al, 2015;Lidberg et al, 1978;Loney et al, 2006).…”

Anxiety associated with perceived uncontrollable stress enhances expectations of environmental volatility and impairs reward learning.

“…Better than expected outcomes elicit positive prediction errors, while worse than expected outcomes elicit negative prediction errors, thus updating estimated action values to support optimal choices (Rigoli et al, 2016;Schultz, 2016;Schultz et al, 1997). Stress can interfere with this learning process, damaging reward maximization (Berghorst et al, 2013;Carvalheiro et al, 2021a;2021b;de Berker et al, 2016;Morgado et al, 2015;Pizzagalli, 2014;Starcke & Brand, 2012;Voulgaropoulou et al, 2021) and impairing avoidance of aversive outcomes (Raio et al, 2017). For example, anticipating threat of shock impairs performance in probabilistic reward learning tasks (Bogdan et al, 2011;Bogdan & Pizzagalli, 2006;Paret & Bublatzky, 2020), and acute stress dampens neural activation associated with reward sensitivity in ventromedial prefrontal cortex (vmPFC) and dorsal striatum (Porcelli et al, 2012).…”