An encounter frequency account of how experience affects likelihood estimation

Abstract

632People are constantly faced with uncertainty. We can only estimate how likely we are to catch the flu this season, to experience medication side effects, or to miss the morning train. To judge how likely an event is to occur, information may be recalled from a variety of sources. In this article, we discuss such sources as belonging to one of two categories: statistical or individual. We aim to show that this distinction is useful when predicting how people will judge the likelihood of an event's occurrence.We start with a familiar result-that is, people's tendency to overlook relevant statistical information. Fong, Krantz, and Nisbett (1986) demonstrated this phenomenon when they asked subjects to make judgments about a number of hypothetical problems in which statistical principles were relevant. For example, subjects read about the Caldwells, a couple trying to decide which of two car brands, Saab or Volvo, is better. Subjects read that both the Consumer Reports car experts and Consumer Reports readers slightly favored Volvos over Saabs. Then they read that the Caldwells have two Saab-owning friends and one Volvo-owning friend. The two Saab owners had few problems with their cars, whereas the Volvo owner describes negative experiences with his car. When asked to evaluate scenarios such as these, fewer than half of subjects' responses included a statistical explanation. Clearly, at least to psychologists, sample size is a relevant concept in this example. The information from Consumer Reports is based on many more people's experiences than that learned from the Caldwells' three friends.Overlooking sample size when comparing information from statistical and personal sources is related to Nisbett and Ross's (1980) "man who," or, more recently, "person who" (Stanovich, 2001), reasoning. "Person who" reasoning occurs when one discounts statistical information in light of a conflicting personal anecdote. For example, one may doubt a report stating that employment is on the rise, because he knows a "person who" can't find a job. This type of reasoning is problematic, because it runs counter to the law of large numbers. According to both the weak and strong versions of the law, a population mean is generally better approximated by a large, rather than a small, sample from a population. Sedlmeier and Gigerenzer (1997) argued that other mathematical theorems, rather than the law of large numbers, are more appropriate for justifying sample size use as normative. Nevertheless, the overall picture is the same: Sample size is a normatively relevant factor, but it is commonly overlooked. In the Volvo-Saab scenario, there is more reason to believe that the Consumer Reports data provide a reliable estimate of car soundness, because they are based on a larger sample than the information provided by the Caldwells' friends. From this viewpoint, the Fong et al. (1986) subjects should have explained that the Volvo is a better choice than the Saab because there is more evidence to support this hypothesis than the other way aroun...

Statistical judgments are influenced by the implied likelihood that samples represent the same population

“…When various other subdivisions of numeracy scores were tested, effects of numeracy and interactions between numeracy and spread and between numeracy and parity were still seen. In line with previous work (Obrecht et al, 2009), these findings suggest that more numerate individuals are more strongly influenced by numerical factors that affect probability than are less numerate individuals. Interestingly, another ANOVA found no effect of having taken a statistics course when this factor, rather than numeracy, was included as the betweensubjects variable, F(1, 215) = .01, p > .9.…”

“…Participants at the highest and high-middle numeracy levels had mean scaled percent responses that were most consistent with the weighted mean model (.686, SE = .079, and .551, SE = .073, respectively), while participants at the middle and low-middle numeracy levels had responses that were most consistent with the unweighted mean model (.415,SE = .074,and .176,SE = .068,respectively). This suggests that, in accordance with previous results (Obrecht et al, 2009), more numerate participants used sample size more extensively. However, scaled responses of the participants at the lowest numeracy level (.364, SE = .087) were more consistent with the weighted mean model than were those in the low-middle group.…”

“…Consequently, a person who varies the weight they give to sample size on the basis of this certainty would give less weight to sample size than would be considered normative. This may explain prior findings that sample size is underweighted relative to normative standards (e.g., Obrecht, 2010), as well as findings that sample size is given little or no weight when anecdotes or individual cases are presented along with summary data (Obrecht et al, 2009;Ubel et al, 2001). If laypeople think that presenting samples separately implies that samples come from separate subpopulations, "underweighting" sample size is justifiable.…”

“…Furthermore, given the findings of Experiment 1 and 2, one might predict that laypeople would be particularly likely to conclude that samples come from different subpopulations when numeric information indicates that the data are less likely to be drawn randomly from the same population. To illustrate, consider Obrecht et al (2009). In this study, participants were told that a report found that 30 out of 1,000 radios tested (3%) broke within a year and also were given statements from radio owners of whom 3 out of 4 or 12 out of 16 claimed that the product broke.…”

“…Also, mean difference has a stronger effect on people's judgments than does sample size (Obrecht et al, 2007). Furthermore, competing factors such as anecdotal descriptions and encounter frequency may lead to sample size being overlooked (Nisbett et al, 1983;Obrecht, Chapman & Gelman, 2009;Ubel, Jepson & Baron, 2001). Additionally, as illustrated by Kahneman and Tversky's (1972) classic "hospital problem," many people fail to recognize that larger sample sizes will reduce the chance that sample means will differ from a population mean (but see Evans & Dusior, 1977;Sedlmeier, 1998).…”