Natural statistics support a rational account of confidence biases

Tw, Webb; Miyoshi, Kiyofumi; Ty, So; Rajananda, Sivananda; H, Lau

doi:10.1101/2021.09.28.462081

Cited by 9 publications

(17 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The present findings suggest that the observation of m-ratio ≥ 1 is not as unnatural as previously considered, alleviating the need to presuppose the existence of metacognitive inefficiency as a default hypothesis. In fact, there is no intrinsic reason within the realm of SDT that restricts m-ratio less than 1, and our previous studies have indeed demonstrated that the pattern of m-ratio ≥ 1 naturally emerges as a consequence of our metacognitive system being adapted to statistical structures of the world (see Miyoshi & Lau, 2020 ; Webb, Miyoshi, So, Rajananda, & Lau, 2021 ).…”

Section: Discussionmentioning

confidence: 96%

On the assumptions behind metacognitive measurements: Implications for theory and practice

2022

Self Cite

View full text Add to dashboard Cite

Theories of visual confidence have largely been grounded in the gaussian signal detection framework. This framework is so dominant that idiosyncratic consequences from this distributional assumption have remained unappreciated. This article reports systematic comparisons of the gaussian signal detection framework to its logistic counterpart in the measurement of metacognitive accuracy. Because of the difference in their distribution kurtosis, these frameworks are found to provide different perspectives regarding the efficiency of confidence rating relative to objective decision (the logistic model intrinsically gives greater meta-dʹ/dʹ ratio than the gaussian model). These frameworks can also provide opposing conclusions regarding the metacognitive inefficiency along the internal evidence continuum (whether meta-dʹ is larger or smaller for higher levels of confidence). Previous theories developed on these lines of analysis may need to be revisited as the gaussian and logistic metacognitive models received somewhat equivalent support in our quantitative model comparisons. Despite these discrepancies, however, we found that across-condition or across-participant comparisons of metacognitive measures are relatively robust against the distributional assumptions, which provides much assurance to conventional research practice. We hope this article promotes the awareness for the significance of hidden modeling assumptions, contributing to the cumulative development of the relevant field.

show abstract

Section: Discussionmentioning

confidence: 96%

On the assumptions behind metacognitive measurements: Implications for theory and practice

2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…In general, the purpose of this article is not to make specific judgments about the merits of one particular model of metacognitive monitoring over another. Such arguments have been the focus of much debate in the metacognitive literature, and we point the interested reader to these works (Fleming & Daw, 2017; Khalvati et al, 2021; Rahnev et al, 2020; Shekhar & Rahnev, 2022; Webb et al, 2021; Yeung & Summerfield, 2012). Rather, the intention of this article is to probe the consequences of what it means to posses more complex forms of confidence, however they might arise, for metacognitive information search and control more generally.…”

mentioning

confidence: 99%

Metacognitive computations for information search: Confidence in control.

Schulz¹,

Fleming²,

Dayan³

2023

Psychological Review

View full text Add to dashboard Cite

The metacognitive sense of confidence can play a critical role in regulating decision making. In particular, a lack of confidence can justify the explicit, potentially costly, instrumental acquisition of extra information that might resolve uncertainty. Human confidence is highly complex, and recent computational work has suggested a statistically sophisticated tapestry behind the information that governs both the making and monitoring of choices. However, the consequences of the form of such confidence computations for search have yet to be understood. Here, we reveal extra richness in the use of confidence for information seeking by formulating joint models of action, confidence, and information search within a Bayesian and reinforcement learning framework. Through detailed theoretical analysis of these models, we show the intricate normative downstream consequences for search arising from more complex forms of metacognition. For example, our results highlight how the ability to monitor errors or general metacognitive sensitivity impact seeking decisions and can generate diverse relationships between action, confidence, and the optimal search for information. We also explore whether empirical search behavior enjoys any of the characteristics of normatively derived prescriptions. More broadly, our work demonstrates that it is crucial to treat metacognitive monitoring and control as closely linked processes.

show abstract

“…However, critically, such dissociations can also be explained in several other ways, such as by assuming that increasing both the signal and noise in a stimulus affects both the signal and variability in the internal evidence distributions. In fact, a recent study showed that high confidence for high-intensity stimuli is observed even in neural networks trained to take all evidence into account (Webb et al, 2021). Therefore, we suggest that empirical confidence-accuracy dissociations where high-intensity stimuli with matched d ′ produce higher confidence rating be renamed to “high-intensity–high-confidence” effect.…”

Section: Discussionmentioning

confidence: 99%

How do humans give confidence? A comprehensive comparison of process models of perceptual metacognition.

Shekhar,

Rahnev

2024

Journal of Experimental Psychology: General

View full text Add to dashboard Cite

Humans have the metacognitive ability to assess the accuracy of their decisions via confidence judgments. Several computational models of confidence have been developed but not enough has been done to compare these models, making it difficult to adjudicate between them. Here, we compare 14 popular models of confidence that make various assumptions, such as confidence being derived from postdecisional evidence, from positive (decision-congruent) evidence, from posterior probability computations, or from a separate decision-making system for metacognitive judgments. We fit all models to three large experiments in which subjects completed a basic perceptual task with confidence ratings. In Experiments 1 and 2, the best-fitting model was the lognormal meta noise (LogN) model, which postulates that confidence is selectively corrupted by signal-dependent noise. However, in Experiment 3, the positive evidence (PE) model provided the best fits. We evaluated a new model combining the two consistently best-performing models-LogN and the weighted evidence and visibility (WEV). The resulting model, which we call logWEV, outperformed its individual counterparts and the PE model across all data sets, offering a better, more generalizable explanation for these data. Parameter and model recovery analyses showed mostly good recoverability but with important exceptions carrying implications for our ability to discriminate between models. Finally, we evaluated each model's ability to explain different patterns in the data, which led to additional insight into their performances. These results comprehensively characterize the relative adequacy of current confidence models to fit data from basic perceptual tasks and highlight the most plausible mechanisms underlying confidence generation. Public Significance StatementSeveral process models have attempted to describe the computations that underlie metacognition in humans. However, due to lack of systematic, widespread comparisons between these models, there is no consensus on what mechanisms best characterize the process of confidence generation. In this study, we tested 14 popular models of metacognition on three large data sets from basic perceptual tasks, using multiple quantitative as well as qualitative metrics. Our results highlight two mechanisms as the most plausible, generalizable features of confidence-the selective corruption of confidence by signal-dependent metacognitive noise and a heuristic strategy that uses stimulus visibility to estimate confidence. Analyzing the qualitative patterns of confidence generated by the models provides additional insights into each model's success or failure. Our results also help to establish a comprehensive framework for model comparisons that can guide future efforts.

show abstract

Natural statistics support a rational account of confidence biases

Cited by 9 publications

References 80 publications

On the assumptions behind metacognitive measurements: Implications for theory and practice

On the assumptions behind metacognitive measurements: Implications for theory and practice

Metacognitive computations for information search: Confidence in control.

How do humans give confidence? A comprehensive comparison of process models of perceptual metacognition.

Contact Info

Product

Resources

About