Calibration of Probabilities: The State of the Art to 1980

Lichtenstein, Sarah; Fischhoff, Baruch; Phillips, Lawrence D.

doi:10.21236/ada101986

Cited by 325 publications

(477 citation statements)

References 3 publications

Supporting

Mentioning

449

Contrasting

Unclassified

Order By: Relevance

“…First, we observed that confidence strongly predicted choice accuracy (Figure 2B, E, S3), as had been shown previously (Jastrow and Peirce, 1884; Lichtenstein et al, 1981). Second, we found that self-reported confidence increased with evidence discriminability for correct trials, but decreased with discriminability on errors, in accordance with a counterintuitive prediction of statistical confidence (Figure 2C, F).…”

Section: Resultssupporting

confidence: 87%

“…This way we sought to sidestep complex calibration issues related to explicit estimation of a probability (Juslin et al, 2000; Lichtenstein et al, 1981) and risk-sensitivity (Fleming and Dolan, 2010). Our confidence report method contrasts with other assays where choice and confidence are reported with the same motor response (Bahrami et al, 2012; Kiani et al, 2014; Zylberberg et al, 2012).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Signatures of a Statistical Computation in the Human Sense of Confidence

Sanders

Hangya

Kepecs

2016

Neuron

238

409

View full text Add to dashboard Cite

Summary Human confidence judgments are thought to originate from metacognitive processes that provide a subjective assessment about one’s beliefs. Alternatively, confidence is framed in mathematics as an objective statistical quantity: the estimated probability that a chosen hypothesis is correct. Despite similar terminology, it remains unclear whether the subjective feeling of confidence is related to the objective, statistical computation of confidence. To address this, we collected confidence reports from humans performing perceptual and knowledge-based psychometric decision tasks. We observed two counterintuitive patterns relating confidence to choice and evidence: apparent overconfidence in choices based on uninformative evidence, and for erroneous choices, that confidence decreased with increasing evidence strength. We show that these patterns lawfully arise when statistical confidence qualifies a decision. Furthermore, statistical confidence quantitatively accounted for human confidence in our tasks without necessitating heuristic operations. Accordingly, we suggest that the human feeling of confidence originates from a mental computation of statistical confidence.

show abstract

Section: Resultssupporting

confidence: 87%

Section: Discussionmentioning

confidence: 99%

Signatures of a Statistical Computation in the Human Sense of Confidence

Sanders

Hangya

Kepecs

2016

Neuron

238

409

View full text Add to dashboard Cite

show abstract

“…Traditional research on overconfidence applies to contexts where a DM’s reported chances of answering a question correct are larger than their actual performance (see Lichtenstein, Fischhoff, & Phillips, 1982). In these studies, confidence calibration has a statistical interpretation based on observing the true outcomes.…”

Section: Discussionmentioning

confidence: 99%

Global–Local Incompatibility: The Misperception of Reliability in Judgment Regarding Global Variables

Broomell

2020

Cognitive Science

View full text Add to dashboard Cite

A number of important decision domains, including decisions about hiring, global warming, and weather hazards, are characterized by a global-local incompatibility. These domains involve variables that cannot be observed by a single decision maker (DM) and require the integration of observations from locally available information cues. This paper presents a new bifocal lens model that describes how the structure of the environment can lead to a unique form of overconfidence when generalizing the reliability of the local environment to a global scale. When the local environment does not reliably reflect the global environment, they are incompatible. While local perspectives vary across DMs, global-local incompatibility can be understood using the structure of classical test theory as the difference between (a) perceived estimates of the reliability derived from the local environment and (b) the true reliability of the local environment. I model global-local incompatibility as the difference between the true and estimated reliability when the assumptions of classical test theory are violated. Using a series of case studies and an empirical study, I demonstrate the widespread utility of this framework, and I conclude by discussing implications for cognitive-ecological theory, risk communication, and overconfidence.

show abstract

“…Effective resolution of public policy problems is also hindered by the overconfidence of policymakers. Overconfidence among decision makers is widely documented in the psychology and decision science literature (Lichtenstein and Fischhoff, 1977; Lichtenstein et al , 1982). Individuals tend to be overconfident in their decisions when dealing with moderate or extremely difficult questions, expressing 90 percent subjective confidence intervals that in fact only contain the true value about 30–60 percent of the time (Bazerman, 1994).…”

Section: “The Problems” Of Public Policy Problemsmentioning

confidence: 99%

How small system dynamics models can help the public policy process

Ghaffarzadegan

Lyneis²,

Richardson

2010

System Dynamics Review

299

169

View full text Add to dashboard Cite

Public policies often fail to achieve their intended result because of the complexity of both the environment and the policy making process. In this article, we review the benefits of using small system dynamics models to address public policy questions. First we discuss the main difficulties inherent in the public policy making process. Then, we discuss how small system dynamics models can address policy making difficulties by examining two promising examples: the first in the domain of urban planning and the second in the domain of social welfare. These examples show how small models can yield accessible, insightful lessons for policy making stemming from the endogenous and aggregate perspective of system dynamics modeling and simulation.

show abstract

Calibration of Probabilities: The State of the Art to 1980

Cited by 325 publications

References 3 publications

Signatures of a Statistical Computation in the Human Sense of Confidence

Signatures of a Statistical Computation in the Human Sense of Confidence

Global–Local Incompatibility: The Misperception of Reliability in Judgment Regarding Global Variables

How small system dynamics models can help the public policy process

Contact Info

Product

Resources

About