Subjective Confidence as a Monitor of the Replicability of the Response

Abstract

Confidence is commonly assumed to monitor the accuracy of responses. However, intriguing results, examined in the light of philosophical discussions of epistemic justification, suggest that confidence actually monitors the reliability of choices rather than (directly) their accuracy. The focus on reliability is consistent with the view that the construction of truth has much in common with the construction of reality: extracting reliable properties that afford prediction. People are assumed to make a binary choice by sampling cues from a “collective wisdomware,” and their confidence is based on the consistency of these cues, in line with the self-consistency model. Here, however, I propose that internal consistency is taken to index the reliability of choices themselves—the likelihood that they will be repeated. The results of 10 studies using binary decisions from different domains indicated that confidence in a choice predicts its replicability both within individuals and across individuals. This was so for domains for which choices have a truth value and for those for which they do not. For the former domains, differences in replicability mediated the prediction of accuracy whether confidence was diagnostic or counterdiagnostic of accuracy. Metatheoretical, methodological, and practical implications are discussed.

The Effect of Source Reliability and Information Credibility on Judgments of Information Quality in Intelligence Analysis

“…Exploration of the confidence-reliability relation would appear to be supported by Koriat's (2024) recent suggestion that one's confidence level in a given response reflects the general replicability of that response. That is, we would expect levels of uncertainty or confidence level in one's rating of information quality to also be predictive of the reliability or consistency within ratings.…”