Measurement Error and the Hot Hand

Misses in “Hot Hand” Research

“…Furthermore, I can determine the percentage of the times in which the test detects statistical significance. The ordinary least squares regression of the success of the current shot on the prior shot produces a consistent estimate of the first-order autocorrelation (Stone, 2012). Furthermore, the test of autocorrelation is redundant with the runs test used in prior studies (Wardrop, 1999).…”

“…But this definition misses a key point that a made shot (or a win) is not necessarily evidence for being in the ''hot'' state (or for having momentum). Perhaps a better definition for the hot hand, used implicitly by Wardrop (1999) and formally stated by Stone (2012), is that ''the hot hand exists if the probability of a hit is positively associated with the probability of a hit for the next shot, and that the magnitude of the hot hand phenomenon is larger when this association is greater.'' It is quite plausible that the near-consensus finding of no hot hand in basketball is the true answer.…”

Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Numbers

“…Miller and Sanjurjo () showed that the runs and serial correlation tests, along with the conditional probability test for , all amount to roughly the same test, and moreover, that they are not sufficiently powered to identify hot hand shooting. The reason why is due to measurement error: the act of hitting a single shot is only a weak signal of a change in a player's underlying probability of success, which leads to an attenuation bias in the estimate of the increase in the probability of success associated with entering the hot state (see Appendix B and Stone ()'s work on measurement error when estimating autocorrelation in ability). The test of variation in four‐shot windows is even less powered than the aforementioned tests ( Wardrop (), Miller and Sanjurjo ()).…”

“… The in‐game free throw data that GVT analyzed (Study 3: Celtics, 1980–81, 1981–82 seasons: 9 players), while arguably controlled, are not ideal for the study of hot hand shooting for a number of reasons: (i) hitting the first shot in a pair of isolated shots is not typically regarded by fans and players as hot hand shooting ( Koehler and Conley ()), presumably due to the high prior probability of success (), (ii) hitting a single shot is a weak signal of a player's underlying state, which can lead to severe measurement error ( Stone (), Arkes ()), (iii) it is vulnerable to an omitted variable bias, as free throw pairs are relatively rare, and shots must be aggregated across games and seasons in order to have sufficient sample size ( Miller and Sanjurjo ()). In any event, subsequent studies of free throw data have found evidence that is inconsistent with the conclusions that GVT drew from the Celtics' data ( Wardrop (), Arkes (), Yaari and Eisenmann (), Aharoni and Sarig (), Goldman and Rao (), Miller and Sanjurjo ()). …”

How should “hot” players in basketball be defended? The use of fast-and-frugal heuristics by basketball coaches and players in response to streakiness

“…Many arguments centred on the question whether the traditional statistical tests employed in the hot-hand literature, i.e., the runs test and autocorrelations, have been adequate and sufficiently powerful to detect the hot hand were it to exist. Accordingly, Stone (2012) claimed that it is not appropriate to use autocorrelation as a test of detecting the hot hand and developed a model by differentiating between shot results and shot probabilities. Specifically, he assumed that the probabilities of hitting a shot may be higher during hot streaks, e.g., 70% instead of 50%, but that even a shot attempt with an increased success probability of 70% may result in a miss.…”