Simultaneous over- and underconfidence: The role of error in judgment processes.

Erev, Ido; Wallsten, Thomas S.; Budescu, David V.

doi:10.1037/0033-295x.101.3.519

Cited by 607 publications

(588 citation statements)

References 36 publications

(51 reference statements)

Supporting

Mentioning

569

Contrasting

Unclassified

Order By: Relevance

“…For the last day of forecasting, it was 0.10. 3 This represents impressively good calibration, especially because the forecasting questions were selected to be difficult and overconfidence tends to be greatest on difficult questions (Erev et al 1994, Juslin et al 2000, Klayman et al 1999.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Confidence Calibration in a Multi-year Geopolitical Forecasting Competition

Swift

Minster²,

Mellers³

et al. 2016

Preprint

View full text Add to dashboard Cite

This research examines the development of confidence and accuracy over time in the context of forecasting. Although overconfidence has been studied in many contexts, little research examines its progression over long periods of time or in consequential policy domains. This study employs a unique data set from a geopolitical forecasting tournament spanning three years in which thousands of forecasters predicted the outcomes of hundreds of events. We sought to apply insights from research to structure the questions, interactions, and elicitations to improve forecasts. Indeed, forecasters’ confidence roughly matched their accuracy. As information came in, accuracy increased. Confidence increased at approximately the same rate as accuracy, and good calibration persisted. Nevertheless, there was evidence of a small amount of overconfidence (3%), especially on the most confident forecasts. Training helped reduce overconfidence and team collaboration improved forecast accuracy. Together, teams and training reduced overconfidence to 1%. Our results provide reason for tempered optimism regarding confidence calibration and its development over time in consequential field contexts.

show abstract

Section: Resultsmentioning

confidence: 99%

“…There is simply more room to be overconfident on difficult items (on which most people are guessing) than on easy items (that most people get right). This is the so-called hardeasy effect in confidence judgments (Erev et al 1994). Were our forecasting items easy or hard?…”

Section: On the Importance Of Forecastingmentioning

confidence: 99%

Confidence Calibration in a Multi-year Geopolitical Forecasting Competition

Swift

Minster²,

Mellers³

et al. 2016

Preprint

View full text Add to dashboard Cite

show abstract

“…Erev, Wallsten, and Budescu (1994) pointed out that when people have imperfect knowledge of their own performances, the error in their estimates will make those estimates regressive. As a result, people underestimate their performance when it is high (Burson et al, 2006;Krueger & Mueller, 2002).…”

Section: Underestimationmentioning

confidence: 99%

The trouble with overconfidence.

Moore

Healy

2008

Psychological Review

1,852

1,414

View full text Add to dashboard Cite

The authors present a reconciliation of 3 distinct ways in which the research literature has defined overconfidence: (a) overestimation of one's actual performance, (b) overplacement of one's performance relative to others, and (c) excessive precision in one's beliefs. Experimental evidence shows that reversals of the first 2 (apparent underconfidence), when they occur, tend to be on different types of tasks. On difficult tasks, people overestimate their actual performances but also mistakenly believe that they are worse than others; on easy tasks, people underestimate their actual performances but mistakenly believe they are better than others. The authors offer a straightforward theory that can explain these inconsistencies. Overprecision appears to be more persistent than either of the other 2 types of overconfidence, but its presence reduces the magnitude of both overestimation and overplacement.

show abstract

“…According to this new view, judges' responses contain lots of error, but little or no bias. There are several sources of unsystematic error in subjective confidence, ranging from having accidentally had atypical experiences (Juslin, 1994;Soll, 1996) to unreliability in generating a number that goes with one's feeling of confidence (Erev, Wallsten, & Budescu, 1994). The proportion of correct answers is also susceptible to variation depending on which particular questions happen to be sampled.…”

Section: Overconfidence In Interval Estimatesmentioning

confidence: 99%

Overconfidence in Interval Estimates.

Soll

Klayman

2004

Journal of Experimental Psychology: Learning, Memory, and Cogni

348

345

View full text Add to dashboard Cite

Judges were asked to make numerical estimates (e.g., “In what year was the first flight of a hot air balloon?”). Judges provided high and low estimates such that they were X% sure that the correct answer lay between them. They exhibited substantial overconfidence: The correct answer fell inside their intervals much less than X% of the time. This contrasts with choices between 2 possible answers to a question, which showed much less overconfidence. The authors show that overconfidence in interval estimates can result from variability in setting interval widths. However, the main cause is that subjective intervals are systematically too narrow given the accuracy of one's information—sometimes only 40% as large as necessary to be well calibrated. The degree of overconfidence varies greatly depending on how intervals are elicited. There are also substantial differences among domains and between male and female judges. The authors discuss the possible psychological mechanisms underlying this pattern of findings.

show abstract

Simultaneous over- and underconfidence: The role of error in judgment processes.

Cited by 607 publications

References 36 publications

Confidence Calibration in a Multi-year Geopolitical Forecasting Competition

Confidence Calibration in a Multi-year Geopolitical Forecasting Competition

The trouble with overconfidence.

Overconfidence in Interval Estimates.

Contact Info

Product

Resources

About