Contextual modulation of value signals in reward and punishment learning

Palminteri, Stefano; Khamassi, Mehdi; Joffily, Mateus; Coricelli, Giorgio

doi:10.1038/ncomms9096

Cited by 275 publications

(566 citation statements)

References 68 publications

Supporting

Mentioning

460

Contrasting

Order By: Relevance

“…We fitted the RELATIVE model proposed by Palminteri et al (2015) to the data of Experiment 1, and generated predictions of two latent variables for Experiments 2, 3, and 4. In particular, we were interested in the unsigned difference between the available option values (|ΔQ t |), and in the context value (V t ).…”

Section: Computational Model-based Analysismentioning

confidence: 99%

“…The local ethical committees approved the studies and the subjects provided written informed consent; see the original publications for additional details (Palminteri et al, 2015(Palminteri et al, , 2016Salvador et al, 2017).…”

Section: Subjectsmentioning

confidence: 99%

“…To capture the trial-by-trial dynamics due to learning-by-feedback, we fitted the RELATIVE model, proposed by Palminteri, Khamassi, Joffily, and Coricelli (2015). This model is based on a simple Q-learning model (Sutton and Barto, 1998), but allows separate learning rate parameters for outcomes of chosen and forgone options, and includes a contextual module, so that option values are updated relative to the learned value of the choice context.…”

Section: Rl Architecture and Fittingmentioning

confidence: 99%

“…In a series of recent studies, Palminteri and colleagues (Palminteri et al, 2015(Palminteri et al, , 2016(Palminteri et al, , 2017a) developed a RL paradigm where they orthogonally manipulated two important contextual features: feedback information and outcome valence. Feedback information was modulated by showing (i.e., complete feedback) or not showing (i.e., partial feedback) the outcome associated with the unchosen option.…”

Section: Introductionmentioning

confidence: 99%

“…Then, we fitted a previously proposed RL model (Palminteri et al, 2015). Finally, we moved to the SSM framework: Specifically, we used a hierarchical Bayesian version of the standard diffusion decision model (DDM, Ratcliff, 1978) and tested the effects of the contextual factors (i.e., feedback information and valence) on the model parameters (i.e., drift-rate, threshold, and non-decision time).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Decomposing the effects of context valence and feedback information on speed and accuracy during reinforcement learning: A meta-analytical approach using diffusion decision modeling

Palminteri¹,

Fontanesi²,

Lebreton³

2018

Preprint

Self Cite

View full text Add to dashboard Cite

When humans and animals learn by trial-and-error to select the most advantageous action, the progressive increase in action selection accuracy due to learning is typically accompanied by a decrease in the time needed to execute this action. Both choice and response time (RT) data can thus provide information about decision and learning processes. However, traditional reinforcement learning (RL) models focus exclusively on the increase in choice accuracy and ignore RTs. Consequently, they neither decompose the interactions between choices and RTs, nor investigate how these interactions are influenced by contextual factors. However, at least in the field of perceptual decision-making, such interactions have proven to be important to dissociate between the underlying processes. Here, we analyzed such interactions in behavioral data from four experiments, which feature manipulations of two factors: outcome valence (gains vs. losses) and feedback information (partial vs. complete feedback). A Bayesian metaanalysis revealed that these contextual factors differently affect RTs and accuracy. To disentangle the processes underlying the observed behavioral patterns, we jointly fitted choices and RTs across all experiments with a single, Bayesian, hierarchical diffusion decision model (DDM). In punishmentavoidance contexts, compared to reward-seeking contexts, participants consistently slowed down without any loss of accuracy. The DDM explained these effects by shifts in the non-decision time and threshold parameters. The reduced motor facilitation may represent the basis of Pavlovian-toinstrumental transfer biases, while the increased cautiousness might be induced by the expectation of losses and be consistent with the loss attention framework.

show abstract

Section: Computational Model-based Analysismentioning

confidence: 99%

Section: Subjectsmentioning

confidence: 99%

Section: Rl Architecture and Fittingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Decomposing the effects of context valence and feedback information on speed and accuracy during reinforcement learning: A meta-analytical approach using diffusion decision modeling

Palminteri¹,

Fontanesi²,

Lebreton³

2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Upward and downward comparisons across monetary and status domains

Yaple

2020

Human Brain Mapping

View full text Add to dashboard Cite

The ability to accurately infer one's place with respect to others is crucial for social interactions. Individuals tend to evaluate their own actions and outcomes by comparing themselves to others in either an upward or downward direction. We performed two fMRI meta-analyses on monetary (n = 39; 1,231 participants) and status (n = 23; 572 participants) social comparisons to examine how domain and the direction of comparison can modulate neural correlates of social hierarchy. Overall, both status and monetary downward comparisons activated regions associated with reward processing (striatum) while upward comparisons yielded loss-related activity. These findings provide partial support for the common currency hypothesis in that downward and upward comparisons from both monetary and status domains resemble gains and losses, respectively. Furthermore, status upward and monetary downward comparisons revealed concordant orbitofrontal cortical activity, an area associated with evaluating the value of goals and decisions implicated in both lesion and empirical fMRI studies investigating social hierarchy. These findings may offer new insight into how people relate to individuals with higher social status and how these social comparisons deviate across monetary and social status domains.

show abstract

Effective connectivity underlying reward‐based executive control

Hippmann

Tzvi

Göttlich

et al. 2021

Human Brain Mapping

View full text Add to dashboard Cite

Motivational influences on cognitive control play an important role in shaping human behavior. Cognitive facilitation through motivators such as prospective reward or punishment is thought to depend on regions from the dopaminergic mesocortical network, primarily the ventral tegmental area (VTA), inferior frontal junction (IFJ), and anterior cingulate cortex (ACC). However, how interactions between these regions relate to motivated control remains elusive. In the present functional magnetic resonance imaging study, we used dynamic causal modeling (DCM) to investigate effective connectivity between left IFJ, ACC, and VTA in a task‐switching paradigm comprising three distinct motivational conditions (prospective monetary reward or punishment and a control condition). We found that while prospective punishment significantly facilitated switching between tasks on a behavioral level, interactions between IFJ, ACC, and VTA were characterized by modulations through prospective reward but not punishment. Our DCM results show that IFJ and VTA modulate ACC activity in parallel rather than by interaction to serve task demands in reward‐based cognitive control. Our findings further demonstrate that prospective reward and punishment differentially affect neural control mechanisms to initiate decision‐making.

show abstract

Contextual modulation of value signals in reward and punishment learning

Cited by 275 publications

References 68 publications

Decomposing the effects of context valence and feedback information on speed and accuracy during reinforcement learning: A meta-analytical approach using diffusion decision modeling

Decomposing the effects of context valence and feedback information on speed and accuracy during reinforcement learning: A meta-analytical approach using diffusion decision modeling

Upward and downward comparisons across monetary and status domains

Effective connectivity underlying reward‐based executive control

Contact Info

Product

Resources

About