Don't Think, Just Feel the Music: Individuals with Strong Pavlovian-to-Instrumental Transfer Effects Rely Less on Model-based Reinforcement Learning

Sebold, Miriam; Schad, Daniel J.; Nebe, Stephan; Garbusow, Maria; Jünger, Elisabeth; Kroemer, Nils B.; Kathmann, Norbert; Zimmermann, U.; Smolka, Michael N.; Rapp, Michael A.; Heinz, Andreas; Huys, Quentin J. M.

doi:10.1162/jocn_a_00945

Cited by 43 publications

(43 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, previous research has shown that general PIT effects depend mainly on the model-free system, which accumulates values through experience (i.e., temporal difference learning; Dolan and Dayan, 2013; Dayan and Berridge, 2014; Garbusow et al, 2014; Sebold et al, 2016). Model-free actions are by definition not executed with respect to the identity of the outcome.…”

Section: Discussionmentioning

confidence: 99%

“…This experimental procedure is called Pavlovian-to-instrumental transfer (PIT). The PIT phenomenon has been widely investigated in both animals (for review see Holmes et al, 2010) and humans (Bray et al, 2008; Talmi et al, 2008; Huys et al, 2011, 2016; Prévost et al, 2012; Lewis et al, 2013; Watson et al, 2014; Cartoni et al, 2015; Garofalo and di Pellegrino, 2015; Lovibond et al, 2015; Sebold et al, 2016; Quail et al, 2017) making this a useful model for translational research and in addressing our questions about potential reward-type specific influences on behavior.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Monetary, Food, and Social Rewards Induce Similar Pavlovian-to-Instrumental Transfer Effects

Lehner

Balsters²,

Herger³

et al. 2017

Front. Behav. Neurosci.

View full text Add to dashboard Cite

Multiple types of reward, such as money, food or social approval, are capable of driving behavior. However, most previous investigations have only focused on one of these reward classes in isolation, as such it is not clear whether different reward classes have a unique influence on instrumental responding or whether the subjective value of the reward, rather than the reward type per se, is most important in driving behavior. Here, we investigate behavior using a well-established reward paradigm, Pavlovian-to-instrumental transfer (PIT), and three different reward types: monetary, food and social rewards. The subjective value of each reward type was matched using a modified Becker-DeGroot-Marschak (BDM) auction where subjective reward value was expressed through physical effort using a bimanual grip force task. We measured the influence of reward-associated stimuli on how participants distributed forces between hands when reaching a target effort range on the screen bimanually and on how much time participants spent in this target range. Participants spent significantly more time in the target range (15% ± 2% maximal voluntary contraction) when a stimulus was presented that was associated with a reward used during instrumental conditioning or Pavlovian conditioning compared to a stimulus associated with a neutral outcome (i.e., general PIT). The strength of the PIT effect was modulated by subjective value (i.e., individuals who showed a stronger PIT effect rated the value of rewards more highly), but not by reward type, demonstrating that stimuli of all reward types were able to act as appetitive reinforcers and influenced instrumental responding, when matched to the same subjective reward value. This is the first demonstration that individually matched monetary, food and social rewards are equally effective as appetitive reinforcers in PIT. These findings strengthen the hypotheses that the subjective value is crucial for how much reward-associated stimuli influence behavior.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Monetary, Food, and Social Rewards Induce Similar Pavlovian-to-Instrumental Transfer Effects

Lehner

Balsters²,

Herger³

et al. 2017

Front. Behav. Neurosci.

View full text Add to dashboard Cite

show abstract

“…As previous research has indicated that differences in age could impact on the balance between model-free and model-based control (Eppinger et al, 2013; Sebold et al, 2016), our results could potentially have been confounded by age effects. In order to test this, we performed additional analyses, where we put age (z-scaled) as an additional nuisance regressor in the two previously described linear models.…”

Section: Methodsmentioning

confidence: 82%

“…Ethical approval for the study was obtained in accordance with the Declaration of Helsinki from the Medical Ethics Committees of Charité–Universitätsmedizin Berlin (EA/1/157/11) and Technische Universität Dresden (EK 228072012). Not included were subjects with Axis I psychiatric disorders except nicotine dependence, alcohol abuse and specific phobia according to DSM-IV as measured with the Composite International Diagnostic Interview (CIDI, Wittchen and Pfister, 1997; Jacobi et al, 2013), subjects with DSM-IV personality disorders (SAPAS screening; Moran et al, 2003), and subjects with MRI contraindications (for further details see Sebold et al, 2016). …”

Section: Methodsmentioning

confidence: 99%

How Accumulated Real Life Stress Experience and Cognitive Speed Interact on Decision-Making Processes

Friedel

Sebold

Kuitunen‐Paul

et al. 2017

Front. Hum. Neurosci.

View full text Add to dashboard Cite

Rationale: Advances in neurocomputational modeling suggest that valuation systems for goal-directed (deliberative) on one side, and habitual (automatic) decision-making on the other side may rely on distinct computational strategies for reinforcement learning, namely model-free vs. model-based learning. As a key theoretical difference, the model-based system strongly demands cognitive functions to plan actions prospectively based on an internal cognitive model of the environment, whereas valuation in the model-free system relies on rather simple learning rules from operant conditioning to retrospectively associate actions with their outcomes and is thus cognitively less demanding. Acute stress reactivity is known to impair model-based but not model-free choice behavior, with higher working memory capacity protecting the model-based system from acute stress. However, it is not clear which impact accumulated real life stress has on model-free and model-based decision systems and how this influence interacts with cognitive abilities.Methods: We used a sequential decision-making task distinguishing relative contributions of both learning strategies to choice behavior, the Social Readjustment Rating Scale questionnaire to assess accumulated real life stress, and the Digit Symbol Substitution Test to test cognitive speed in 95 healthy subjects.Results: Individuals reporting high stress exposure who had low cognitive speed showed reduced model-based but increased model-free behavioral control. In contrast, subjects exposed to accumulated real life stress with high cognitive speed displayed increased model-based performance but reduced model-free control.Conclusion: These findings suggest that accumulated real life stress exposure can enhance reliance on cognitive speed for model-based computations, which may ultimately protect the model-based system from the detrimental influences of accumulated real life stress. The combination of accumulated real life stress exposure and slower information processing capacities, however, might favor model-free strategies. Thus, the valence and preference of either system strongly depends on stressful experiences and individual cognitive capacities.

show abstract

“…Through Pavlovian conditioning, any neutral stimulus can 8 become a conditioned stimulus (i.e. a cue) if it has been paired with the effects of the addictive 9 behavior 7 . In addictive disorders, including GD, cues may induce attentional bias, arousal, and 10 craving for the addictive behavior in periods of abstinence 8,9 .…”

Section: Introductionmentioning

confidence: 99%

Cue-induced effects on decision-making distinguish subjects with gambling disorder from healthy controls

Genauck

Andrejević

Brehm

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

While an increased impact of cues on decision-making has been associated with substance dependence, it is yet unclear whether this is also a phenotype of non-substance related addictive disorders, such as gambling disorder. To better understand the basic mechanisms of impaired decision-making in addiction, we investigated whether cue-induced changes in decision-making could distinguish gambling disorder (GD) from healthy control (HC) subjects. We expected that cue-induced changes in gamble acceptance and specifically in loss aversion would distinguish GD from HC subjects.30 GD subjects and 30 matched HC subjects completed a mixed gambles task where gambling and other emotional cues were shown in the background. We used machine learning and classification to carve out the importance of cue-dependency of decision-making and of loss aversion for distinguishing GD from HC subjects.Cross-validated classification yielded an area under the receiver operating curve (AUC-ROC) of 68.9% (p=0.002). Applying the classifier to an independent sample yielded an AUC-ROC of 65.0% (p=0.047). As expected, the classifier used cue-induced changes in gamble acceptance to distinguish GD from HC. Especially increased gambling during the presentation of gambling cues was characteristic of GD subjects. However, unexpectedly, cue-induced changes in loss aversion were irrelevant for distinguishing GD from HC subjects. To our knowledge, this is the first study to investigate the classificatory power of addiction-relevant behavioral task parameters when distinguishing GD from HC subjects. The results indicate that cue-induced changes in decision-making are a characteristic feature of addictive disorders, independent of a substance of abuse.

show abstract

Don't Think, Just Feel the Music: Individuals with Strong Pavlovian-to-Instrumental Transfer Effects Rely Less on Model-based Reinforcement Learning

Cited by 43 publications

References 66 publications

Monetary, Food, and Social Rewards Induce Similar Pavlovian-to-Instrumental Transfer Effects

Monetary, Food, and Social Rewards Induce Similar Pavlovian-to-Instrumental Transfer Effects

How Accumulated Real Life Stress Experience and Cognitive Speed Interact on Decision-Making Processes

Cue-induced effects on decision-making distinguish subjects with gambling disorder from healthy controls

Contact Info

Product

Resources

About