A Simple Remedy for Overprecision in Judgment

Abstract

Overprecision is the most robust type of overconfidence. We present a new method that significantly reduces this bias and offers insight into its underlying cause. In three experiments, overprecision was significantly reduced by forcing participants to consider all possible outcomes of an event. Each participant was presented with the entire range of possible outcomes divided into intervals, and estimated each interval’s likelihood of including the true answer. The superiority of this Subjective Probability Interval Estimate (SPIES) method is robust to range widths and interval grain sizes. Its carryover effects are observed even in subsequent estimates made using the conventional, 90% confidence interval method: judges who first made SPIES judgments considered a broader range of values in subsequent conventional interval estimates as well.

A Behavioral Demonstration of Overconfidence in Judgment

“…We hope that the methodology and the evidence we present here will prove useful in elucidating the underlying sources of this excessive faith in the accuracy of our own judgment. Elicitation methods that call explicit attention to a large range of outcomes can reduce overprecision (see Haran, et al, 2010;Winman, Hansson, & Juslin, 2004), and, in some cases, even cause underprecision (Bolger & Harvey, 1995;Goodwin, 2005). 2 We relied on each person's empirical distribution of errors for each payoff function to identify his or her normative adjustment rather than make parametric assumptions about those distributions, such as normality or invariance across payoff functions.…”

“…The implicit measure was the subjective probability distribution of their errors provided after the first block of point estimates. We expected participants with less confidence to construct wider probability distributions (as measured by the standard deviation of their distributions) than those with more confidence (Haran, et al, 2010).…”

“…Second, after making their point estimates under symmetric payoffs, all participants provided subjective probability distributions of their errors based on the SPIES technique (Haran, Moore, & Morewedge, 2010). They indicated, for every 100 temperature estimates, how many of their errors would fall into each of nine intervals (viz., < -11°C, -10 to -8°C,…, +8 to +10°C, > +11°C).…”

Testing Best Practices to Reduce the Overconfidence Bias in Multi-criteria Decision Analysis

“…This iterative elicitation method resulted in less overconfidence, in comparison to directly produced intervals. Haran, Moore and Morewedge [46] developed another method in which the participants were able to see the entire range of possible intervals, and then asked to estimate the probability that each of these intervals contains the correct answer such that the sum of all probability estimates is amount to one. This method also led people to produce lesser degrees of overconfidence.…”

Format dependent probabilities: An eye-tracking analysis of additivity neglect

“…Such adjustments are sometimes enforced by explicit instructions (Haran, Moore, & Morewedge, 2010), and sometimes encouraged by introducing more subtle "additivity prompts" (Koehler, Brenner, & Tversky, 1997) or "extensional cues" (Kahneman & Tversky, 1996), like presenting the complete set of outcomes jointly to the same participants. However, it can be shown that a joint presentation format reduces, but does not eliminate, additivity neglect (Riege & Teigen, 2013).…”