Decreased transfer of value to action in Tourette syndrome

Schüller, Thomas; Fischer, Adrian G.; Gruendler, Theo O.J.; Baldermann, Juan Carlos; Huys, Daniel; Ullsperger, Markus; Kuhn, Jens

doi:10.1016/j.cortex.2019.12.027

Cited by 23 publications

(25 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Humans have been observed to exhibit confirmatory biases when choosing between stimuli or actions that payout with uncertain probability (Chambon et al, 2019; Palminteri et al, 2017; Schuller et al, 2020). These biases drive participants to update positive outcomes (or those that are better than expected) for chosen options more sharply than negative outcomes, but to reverse this update pattern for the unchosen option.…”

Section: Discussionmentioning

confidence: 99%

“…However, a different pattern of data was observed in follow-up studies in which counterfactual feedback was also offered – i.e., the participants were able to view the payout associated with both chosen and unchosen options. Following a feedback on the unchosen option, larger updates were observed for negative prediction errors (Chambon et al, 2019; Palminteri et al, 2017; Schuller et al, 2020). This is consistent with a confirmatory bias rather than a strictly optimistic bias, whereby belief revision helps to strengthen rather than weaken existing preconceptions about which option may be better.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A normative account of confirmation bias during reinforcement learning

Lefebvre

Summerfield

Bogacz

2020

Preprint

View full text Add to dashboard Cite

12Reinforcement learning involves updating estimates of the value of states and actions on the 13 basis of experience. Previous work has shown that in humans, reinforcement learning exhibits 14 a confirmatory bias: when updating the value of a chosen option, estimates are revised more 15 radically following positive than negative reward prediction errors, but the converse is 16observed when updating the unchosen option value estimate. Here, we simulate performance 17 on a multi-arm bandit task to examine the consequences of a confirmatory bias for reward 18harvesting. We report a paradoxical finding: that confirmatory biases allow the agent to 19 maximise reward relative to an unbiased updating rule. This principle holds over a wide range 20 of experimental settings and is most influential when decisions are corrupted by noise. We 21show that this occurs because on average, confirmatory biases overestimate the value of more 22 valuable bandits, and underestimate the value of less valuable bandits, rendering decisions 23 overall more robust in the face of noise. Our results show how apparently suboptimal learning 24 policies can in fact be reward-maximising if decisions are made with finite computational 25 precision. 26 27

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A normative account of confirmation bias during reinforcement learning

Lefebvre

Summerfield

Bogacz

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…We built onand extendedthe model space proposed in (Lebreton et al, 2019) see Figure 5A. Starting from a simple Q-learning algorithm (ABS model), we designed increasingly complex RL models including 1) a context-dependent learning module (Palminteri et al, 2015); 2) an "asymmetric learning" module (Chambon et al, 2020;Lefebvre et al, 2017;Palminteri et al, 2017a;Schüller et al, 2020); 3) a combination of those two modules (RELASYM model family).…”

Section: Modelling Choices In the Learning And The Transfer Phasementioning

confidence: 99%

“…Further investigations revealed that, in binary choices where outcome information is provided for both chosen and unchosen options, the asymmetry is observed for the chosen option, but reversed for the unchosen option: a higher learning rates are observed for negative, compared to positive, outcome prediction errors. This led to the notion of choice confirmatoryrather than simply optimisticlearning: we seem to overweight information that confirms our current course of action (Chambon et al, 2020;Palminteri et al, 2017a;Schüller et al, 2020). In other terms, learning rates analysis in the context of simple reinforcement learning suggest that subjects behave as if they were neglecting outcomes that disconfirm their current choice, which inevitably leads to an overestimation of the value of the currently chosen option and therefore the probability of being correct.…”

Section: Introductionmentioning

confidence: 97%

SalemGarcia_2021

Garcia¹,

Palminteri²,

Lebreton³

2021

Preprint

View full text Add to dashboard Cite

We systematically misjudge our own performance in simple economic tasks. First, we generally overestimate our ability to make correct choices – a bias called overconfidence. Second, we are more confident in our choices when we seek gains than when we try to avoid losses – a bias we refer to as the valence-induced confidence bias. Strikingly, these two biases are also present in reinforcement-learning contexts, despite the fact that outcomes are provided trial-by-trial and could, in principle, be used to recalibrate confidence judgments online. How confidence biases emerge and are maintained in reinforcement-learning contexts is thus puzzling and still unaccounted for. To explain this paradox, we propose that confidence biases stem from learning biases, and test this hypothesis using data from five experiments, where we concomitantly assessed instrumental choices and confidence judgments, during learning and transfer phases. Our results first show that participants’ choices in both tasks are best accounted for by a reinforcement-learning model featuring context-dependent learning and confirmatory update. We then demonstrate that the complex, biased pattern of confidence judgments elicited during both tasks can be explained by an overweighting of the learned value of the chosen option in the computation of confidence judgments. We finally show that, consequently, the individual learning-model parameters responsible for the learning biases – confirmatory updating and outcome context-dependency – are predictive of the individual metacognitive biases. We conclude suggesting that the metacognitive biases originate from fundamentally biased learning computations.

show abstract

“…Based on previous findings suggesting differences in feedback processing between real and fictive outcomes (Fischer & Ullsperger, 2013;Schuller et al, 2020) (Benjamini & Yekutieli, 2001).…”

Section: Eeg Measurements and Analysesmentioning

confidence: 99%

Feedback-related EEG dynamics separately reflect decision parameters, biases, and future choices

Kirschner

Ullsperger

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Optimal decision making in complex environments requires dynamic learning from unexpected events. To speed up learning, we should heavily weight information that indicates state-action-outcome contingency changes and ignore uninformative fluctuations in the environment. Often, however, unrelated information is hard to ignore and can potentially bias our learning. Here we used computational modelling and EEG to investigate learning behaviour in a modified probabilistic choice task that introduced two types of unexpected events that were irrelevant for optimal task performance, but nevertheless could potentially bias learning: pay-out magnitudes were varied randomly and, occasionally, feedback presentation was enhanced by visual surprise. We found that participants' overall good learning performance was biased by distinct effects of these non-normative factors. On the neural level, these parameters are represented in a dynamic and spatiotemporally dissociable sequence of EEG activity. Later in feedback processing the different streams converged on a central to centroparietal positivity reflecting a final pathway of adaptation that governs future behaviour.

show abstract

Decreased transfer of value to action in Tourette syndrome

Cited by 23 publications

References 32 publications

A normative account of confirmation bias during reinforcement learning

A normative account of confirmation bias during reinforcement learning

SalemGarcia_2021

Feedback-related EEG dynamics separately reflect decision parameters, biases, and future choices

Contact Info

Product

Resources

About