Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Numbers

Abstract

We prove that a subtle but substantial bias exists in a common measure of the conditional dependence of present outcomes on streaks of past outcomes in sequential data. The magnitude of this streak selection bias generally decreases as the sequence gets longer, but increases in streak length, and remains substantial for a range of sequence lengths often used in empirical work. We observe that the canonical study in the influential hot hand fallacy literature, along with replications, are vulnerable to the bias. Upon correcting for the bias we find that the long-standing conclusions of the canonical study are reversed. JEL Classification Numbers: C12; C14; C18;C19; C91; D03; G02.

Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Numbers

“…In Supplemental Material Appendix E.1 ( Miller and Sanjurjo ()) we provide a formula that can be used to calculate for . For the special case of a closed form exists, which we provide in Appendix .…”

“… The expected value of the proportion of successes on trials that immediately follow k consecutive successes, , as a function of the total number of trials n , for different values of k and probabilities of success p , using the formula provided in Supplemental Material Appendix E.1 ( Miller and Sanjurjo ()).…”

“… The histogram and kernel density plot of the (exact) discrete probability distribution of , a single player i with n = 100 and n 1 = 50, using a variant of the formula for the distribution provided in Supplemental Material Appendix E.2 ( Miller and Sanjurjo ()).…”

“… In the context of time series regression, this bias is known as the Hurwicz bias ( Hurwicz ()), which is exacerbated when one introduces fixed effects into a time series model with few time periods ( Nerlove (, ), Nickell ()). In Supplemental Material Appendix F.1 ( Miller and Sanjurjo ()), we use a sampling‐without‐replacement argument to show that in the case of , the streak selection bias, along with finite sample bias for autocorrelation (and time series), are essentially equivalent to: (i) a form of selection bias known in the statistics literature as Berkson's bias, or Berkson's paradox ( Berkson (), Roberts et al ()), and (ii) several classic conditional probability puzzles. …”

Do (Australian) jockeys have hot hands?

“…The results from a follow-up meta-analysis (Avugos et al, 2012) provided sufficient evidence for the authors to find against the existence of hot hands in sport in general. However, Miller and Sanjurjo (2018) have recently uncovered a bias in the common measure of the conditional dependence of present outcomes on streaks of past outcomes in sequential data used in many studies. Once this bias is corrected for some of the negative conclusions from the earlier studies are reversed.…”