Non-replication of association studies: “pseudo-failures” to replicate?

Association between single nucleotide polymorphism-genotype and outcome of patients with chronic lymphocytic leukemia in a randomized chemotherapy trial

“…This failure of replication is not necessarily unexpected, given differences between the two case series. As previously discussed, replication studies face the risk of non-validation because of the lack of generalizability, 23,24 especially when there can be significant differences in individual patients' management. Our discovery cases were UK patients participating in a large phase III clinical trial: each patient had, therefore, received standardized management and treatment.…”

What is the probability of replicating a statistically significant effect?

“…As was noted earlier, experimental power depends on the sample size and level, which are known, and on the effect size, which is unknown. The standard approach is to assume that the true effect size is equal to the effect that was actually observed in the initial experiment (e.g., Gorroochurn et al, 2007;Greenwald et al, 1996;Oakes, 1986;Rosenthal, 1993). From this assumed true effect size, it is straightforward to compute power as a function of sample size and level (see, e.g., Cohen, 1988, for relevant formulas, or Faul, Erdfelder, Lang, & Buchner, 2007, for a computer program that performs such computations).…”

“…Because of the importance of replication probability and the confusion surrounding it, recent articles in numerous disciplines have urged researchers to consider replication probability more carefully (e.g., Cumming, 2008;Cumming & Maillardet, 2006;Gorroochurn, Hodge, Heiman, Durner, & Greenberg, 2007;Greenwald, Gonzalez, Harris, & Guthrie, 1996;Killeen, 2005;Robinson & Levin, 1997;Sohn, 1998). Researchers have been offered formulas with which to compute the probability of replicating their current results, and they have been advised to report the resulting replication probabilities as well as-or even in preference to-more traditional statistical measures (e.g., Greenwald et al, 1996;Killeen, 2005;Psychological Science editorial board, 2005).…”

Key Impact of Vgt1 on Flowering Time Adaptation in Maize: Evidence From Association Mapping and Ecogeographical Information

“…The same study showed that three polymorphisms within Vgt1 were strongly associated with flowering time across a panel of 95 inbred lines. Reproducibility of association studies has proven to be a challenging issue in human genetics (Newton-Cheh and Hirschhorn 2005;Gorroochurn et al 2007) but is not yet well-documented in plants. One exception is the relationship between the Dwarf8 gene and flowering time in maize addressed by Andersen et al (2005) and Camus-Kulandaivelu et al (2006), following the pioneering study by Thornsberry et al (2001).…”