SalemGarcia_2021

Abstract

We systematically misjudge our own performance in simple economic tasks. First, we generally overestimate our ability to make correct choices – a bias called overconfidence. Second, we are more confident in our choices when we seek gains than when we try to avoid losses – a bias we refer to as the valence-induced confidence bias. Strikingly, these two biases are also present in reinforcement-learning contexts, despite the fact that outcomes are provided trial-by-trial and could, in principle, be used to recalibrate confidence judgments online. How confidence biases emerge and are maintained in reinforcement-learning contexts is thus puzzling and still unaccounted for. To explain this paradox, we propose that confidence biases stem from learning biases, and test this hypothesis using data from five experiments, where we concomitantly assessed instrumental choices and confidence judgments, during learning and transfer phases. Our results first show that participants’ choices in both tasks are best accounted for by a reinforcement-learning model featuring context-dependent learning and confirmatory update. We then demonstrate that the complex, biased pattern of confidence judgments elicited during both tasks can be explained by an overweighting of the learned value of the chosen option in the computation of confidence judgments. We finally show that, consequently, the individual learning-model parameters responsible for the learning biases – confirmatory updating and outcome context-dependency – are predictive of the individual metacognitive biases. We conclude suggesting that the metacognitive biases originate from fundamentally biased learning computations.

Implicit Counterfactual Effect in Partial Feedback Reinforcement Learning: Behavioral and Modeling Approach

“…Our main model-space included the standard Q-learning model (SQL), the reference-point model (RP) (Palminteri et al, 2015 ), the difference model (Klein et al, 2017 ), and the hybrid model (Bavard et al, 2018 ). The same analysis was also performed on the extended model-space which, in addition to the previously named models, included the forgetting reinforcement learning model (FQL) (Barraclough et al, 2004 ; Ito and Doya, 2009 ; Katahira, 2015 ; Niv et al, 2015 ; Kato and Morita, 2016 ), the experienced-weighted attraction model (EWA) (Camerer and Hua Ho, 1999 ), the sample-based episodic memory model (SBE) (Bornstein et al, 2017 ), and RelAsym model (Garcia et al, 2021 ; Ting et al, 2021 , Supplementary Tables 2–4 ).…”

“…In this model, it is the asymmetric updating of positive and negative prediction errors that improves the performance by increasing the contrast between option values. Second, the RelAsym model (Garcia et al, 2021 ; Ting et al, 2021 ) which is the combination of the confirmation bias and reference point mechanisms. The RelAsym model by having these two factors, not only has the asymmetric updating advantage (performance advantage) but also is able to explain the contextual effect because of the reference point function it used in its mechanism.…”

“…This RelAsym model consists of two relative value learning component (thorough reference point mechanism) and asymmetric updating component (thorough confirmation bias mechanism; Garcia et al, 2021 ; Ting et al, 2021 ). In the reference-point model, outcomes are context-dependent and causes that options' values to be learned relative to their reference-point.…”