Probability Distributions of Common Repeated Events Are Misestimated.

Abstract

Accurately estimating the prospective probability distribution arising from repeated events with known probabilities, such as the number of heads in ten coin flips, represents a simple aptitude necessary for explicit Bayesian updating and useful in optimal decisions in the face of future uncertainty. Across elicitation methods and decision scenarios, people express beliefs that are systematically biased relative to the actual distribution. Participant beliefs reflect a "wizard-hat" shaped distribution, over-estimating the tails and under-estimating the shoulders of the distribution, relative to the actual bell-curve shape. While experts are relatively more accurate than novices, both show significant bias. The bias is not explained by regression to the mean, random error or participant heterogeneity, and is exacerbated by increasing the number of repeated events. The findings caution against assuming Bayesian belief formation in models of statistical reasoning about explicit prospective beliefs based on repeated events with known probabilities.Goodman & Tenenbaum, 2015;Vul, 2010) and should be simple for a Bayesian decision-maker.Across elicitation methods and decision scenarios, both novices and experts estimate a biased "wizard-hat" shaped subjective binomial distribution. They over-estimated the tails and underestimated the shoulders of the distribution relative to the "gendarme-hat" shaped (Edgeworth's characterization, per Stigler 1999) normative distribution.The bias in estimates is broadly consistent with prior findings documenting errors people make in generating "sampling distributions" (Kahneman & Tversky, 1972;Peterson, DuCharme and Edwards 1968;Wheeler and Beach 1968). However, unlike prior findings, these results cannot be explained by high error variance and regression to the mean, heterogeneity, misunderstanding of general distribution characteristics, or elicitation-specific biases. The findings challenge the emerging view that the human brain is adept at optimal statistical processing for sufficiently simple tasks, with broad implications for people's ability to accurately assess and manage risk. 2.Study 1 MethodAdult participants were recruited from Amazon MTurk to complete an online survey. A target of 900 participants were requested, yielding 867 completed surveys. Records with duplicate IP addresses, or who failed a basic attention check were removed prior to analysis, yielding 821 valid completes. The participants constituted a novice population, with less than half holding a Bachelor's degree and only 5% identifying themselves as knowledgeable in statistics.Participants read a hypothetical scenario, either about one of three equal-probability events which would be repeated for 10 independent trials (coin-flips, survey sampling or soccer kicks), or about the distribution of height (a control task that could be estimated based on memory rather than probabilistic inferences). To test the accuracy of beliefs about the full probability distribution Participants were randomly assigned to one of t...