A simple remedy for overprecision in judgment

Abstract

Overprecision is the most robust type of overconfidence. We present a new method that significantly reduces this bias and offers insight into its underlying cause. In three experiments, overprecision was significantly reduced by forcing participants to consider all possible outcomes of an event. Each participant was presented with the entire range of possible outcomes divided into intervals, and estimated each interval’s likelihood of including the true answer. The superiority of this Subjective Probability Interval Estimate (SPIES) method is robust to range widths and interval grain sizes. Its carryover effects are observed even in subsequent estimates made using the conventional, 90% confidence interval method: judges who first made SPIES judgments considered a broader range of values in subsequent conventional interval estimates as well.

The role of actively open-minded thinking in information acquisition, accuracy, and calibration

“…AOT might also have an influence on other problems in judgment and decision making. The positive effect of information acquisition on confidence interval hit-rate we found is consistent with the findings of Haran et al (2010), whose Subjective Probability Interval Estimate (SPIES) method improved calibration of confidence intervals by preventing judges from ignoring alternative outcomes. It is possible that these results were achieved by making all participants behave as high AOT individuals are naturally inclined to, and make a conscious effort to obtain more relevant information during the estimation process.…”

“…Overconfidence in the accuracy of one's estimates-sometimes called overprecision, to distinguish it from other types of overconfidence (Moore & Healy, 2008)-refers to the discrepancy between the confidence people have in the accuracy of their estimates, predictions, or beliefs and actual accuracy rate. Overconfidence has proven to be robust and difficult to remedy, although some interventions have been partially successful (Haran, Moore, & Morewedge, 2010;Soll & Klayman, 2004;Speirs-Bridge et al, 2010). In this work, we examine cognitive styles and personal-ity dimensions that might be related to performance, and seek an explanation for how they work.…”

Overconfidence over the lifespan

“…Measures . We elicited two full Subjective Probability Interval Estimates (SPIES) distributions (Haran, Moore & Morewedge, 2010) of estimated scores. To do this, we asked participants to estimate the probability that they had obtained each of the eleven possible scores (zero through ten).…”

“…One task was a direct replication of the weight-guessing task in Study 1. As in Study 1's weight-guessing task, we elicited two full SPIES distributions (Haran et al, 2010) of estimated scores from every participant. One asked for the subjective probability distribution (SPD) of the participant's own score.…”

Lay understanding of probability distributions

“…Much is written on the topic of eliciting subjective belief distributions, (e.g., Winkler, 1967;Staël von Holstein, 1971;Van Lenthe, 1993a, b), however it is difficult to judge whether a subjective distribution has been "well elicited" or not. Proxies to deal with this problem have been proposed, for instance: checking how often 90% confidence intervals derived from elicited distributions bracket a correct answer across a number of problems (Haran, Moore, & Morewedge, 2010). Generalizations from this research have been that standard fractiles and probability-based methods elicit confidence intervals that are too narrow (e.g., Lichtenstein et al, 1982).…”