Probabilistic mental models: A Brunswikian theory of confidence.

Abstract

Research on people's confidence in their general knowledge has to date produced two fairly stable effects, many inconsistent results, and no comprehensive theory. We propose such a comprehensive framework, the theory of probabilistic mental models (PMM theory). The theory (a) explains both the overconfidence effect (mean confidence is higher than percentage of answers correct) and the hard-easy effect (overconfidence increases with item difficulty) reported in the literature and (b) predicts conditions under which both effects appear, disappear, or invert. In addition, (c) it predicts a new phenomenon, the confidence-frequency effect, a systematic difference between a judgment of confidence in a single event (i.e., that any given answer is correct) and a judgment of the frequency of correct answers in the long run. Two experiments are reported that support PMM theory by confirming these predictions, and several apparent anomalies reported in the literature are explained and integrated into the present framework.Do people think they know more than they really do? In the last 15 years, cognitive psychologists have amassed a large and apparently damning body of experimental evidence on overconfidence in knowledge, evidence that is in turn part of an even larger and more damning literature on so-called cognitive biases. The cognitive bias research claims that people are naturally prone to making mistakes in reasoning and memory, including the mistake of overestimating their knowledge. In this article, we propose a new theoretical model for confidence in knowledge based on the more charitable assumption that people are good judges of the reliability of their knowledge, provided that the knowledge is representatively sampled from a specified reference class. We claim that this model both predicts new experimental results (that we have tested) and explains a wide range of extant experimental findings on confidence, including some perplexing inconsistencies.Moreover, it is the first theoretical framework to integrate the two most striking and stable effects that have emerged from confidence studies-the overconfidence effect and the hardeasy effect-and to specify the conditions under which these effects can be made to appear, disappear, and even invert. In most recent studies (including our own, reported herein), subThis article was written while Gerd Gigerenzer was a fellow at the Center for Advanced Study in the Behavioral Sciences, Stanford, California. We are grateful for financial support provided by the Spencer Foundation and the Deutsche Foischungsgemeinschaft (DFG 170/2-1).We thank Leda Cosmides, Lorraine Daston, Baruch FischhofF, Jennifer Freyd, Kenneth Hammond, Wolfgang Hell, Sarah Licmenstein, Kathleen Much, John Tooby, Amos Tversky, and an anonymous reviewer for helpful comments on earlier versions of this article.Correspondence concerning this article should be addressed to Gerd Gigerenzer, who is now at the Institut fur Psychologic, Hellbrunnerstrasse 34, Universitat Salzburg, A-5020 Salzburg, Austria.

Overconfidence in Interval Estimates.

“…Gigerenzer et al (1991) asked a similar question following a series of binary choice questions. They found that the average retrospective estimate of the number of correct answers was close to the actual average number correct for a representative sample of questions from the domain.…”

“…Overconfidence may have been the predominant finding in earlier studies because of a tendency for experimenters to construct tests that favored the harder questions in any given domain. The hardest questions in a domain are naturally the most likely to be harder than a reasonable person would expect given his or her knowledge (Gigerenzer et al, 1991;Juslin, 1993;May, 1986).…”

The trouble with overconfidence.

“…Prior research has found more overconfidence on probabilistic than frequentistic estimations of performance (Gigerenzer et al, 1991).…”

“…Brunswikian explanations for Trouble with Overconfidence 18 overconfidence hold that overestimation of task performance is an artifact of the way items were selected. The solution to such selection is to establish a reference class with which people are likely to be familiar, and randomly select judgments from that set (Gigerenzer, Hoffrage, & Kleinbölting, 1991). Our theory would offer two explanations for the tendency for overconfidence to go down when problems are randomly drawn from a familiar reference class.…”

Domain-specific reasoning: Social contracts, cheating, and perspective change

“…Consequently, researchers have tended to ignore that distinction in probability revision tasks. In contrast, for the frequentist interpretation of probability, this distinction is fundamental; and so it seems to be for understanding intuitive reasoning as well (Gigerenzer, 1991b;Gigerenzer, Hoffrage & Kleinbolting, 1991).…”