Decodability of Reward Learning Signals Predicts Mood Fluctuations

Abstract

SummaryOur mood often fluctuates without warning. Recent accounts propose that these fluctuations might be preceded by changes in how we process reward. According to this view, the degree to which reward improves our mood reflects not only characteristics of the reward itself (e.g., its magnitude) but also how receptive to reward we happen to be. Differences in receptivity to reward have been suggested to play an important role in the emergence of mood episodes in psychiatric disorders [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]. However, despite substantial theory, the relationship between reward processing and daily fluctuations of mood has yet to be tested directly. In particular, it is unclear whether the extent to which people respond to reward changes from day to day and whether such changes are followed by corresponding shifts in mood. Here, we use a novel mobile-phone platform with dense data sampling and wearable heart-rate and electroencephalographic sensors to examine mood and reward processing over an extended period of one week. Subjects regularly performed a trial-and-error choice task in which different choices were probabilistically rewarded. Subjects’ choices revealed two complementary learning processes, one fast and one slow. Reward prediction errors [17, 18] indicative of these two processes were decodable from subjects’ physiological responses. Strikingly, more accurate decodability of prediction-error signals reflective of the fast process predicted improvement in subjects’ mood several hours later, whereas more accurate decodability of the slow process’ signals predicted better mood a whole day later. We conclude that real-life mood fluctuations follow changes in responsivity to reward at multiple timescales.

SSRIs modulate asymmetric learning from reward and punishment

“…We suggest the asymmetric effect we highlight can go some ways to explain a gradual evolution of an incremental impact of SSRIs on mood. Recent studies have shown that positive and negative surprise strongly impacts on self-reported affective states (Rutledge et al, 2014;Eldar & Niv, 2015;Eldar et al, 2018). Given a key role of surprise, our results explain how serotonergic intervention can, in principle, influence the affective experience of reinforcement.…”

“…To fit the parameters of the different models to subjects' decisions, we used an iterative hierarchical expectation-maximization procedure across the entire sample, separately for each session (Bishop, 2006;Eldar et al, 2018). We sampled 10 5 random settings of the parameters from predefined prior distributions.…”

“…We compared between models in terms of how well each model accounted for subjects' choices by means of the integrated Bayesian Information Criterion (iBIC (Huys et al, 2012;Eldar et al, 2018)). Here, we estimated the evidence in favour of each model (λ) as the mean likelihood of the model given 10 5 random parameter settings drawn from the fitted group-level priors.…”

“…Learning from reinforcement is attributed to two key components: a prediction error (PE), quantifying a difference between expectation and outcome, and a learning rate, determining the impact of PEs on future reward predictions (Sutton & Barto, 1998). There is evidence that PEs also impact emotional states (Rutledge et al, 2014;Eldar & Niv, 2015;Eldar et al, 2018), where mood depends not only on how well things are going in general but whether things are better or worse than expected. Based on the latter findings it follows that learning asymmetries can, in principle, exert a modulatory effect on mood.…”

A neurocomputational model for intrinsic reward

“…Using experience sampling (27,28) combined with functional neuroimaging, we show that intrinsic and extrinsic rewards contribute to affective dynamics (i.e., happiness). Recent studies demonstrate that computational approaches can quantify consistent relationships between subjective feelings and value-based decision-making (20,21,29,30), including in relation to individual social preferences (31). Here, using the same computational approach, applied in the context of a reinforcement learning task, we show that momentary happiness is influenced by both objectively quantifiable rewards and by intrinsic rewards, where the latter involves experiences with no inherent worth.…”