Statistical correction of the Winner’s Curse explains replication variability in quantitative trait genome-wide association studies

Abstract

Genome-wide association studies (GWAS) have identified hundreds of SNPs responsible for variation in human quantitative traits. However, genome-wide-significant associations often fail to replicate across independent cohorts, in apparent inconsistency with their apparent strong effects in discovery cohorts. This limited success of replication raises pervasive questions about the utility of the GWAS field. We identify all 332 studies of quantitative traits from the NHGRI-EBI GWAS Database with attempted replication. We find that the majority of studies provide insufficient data to evaluate replication rates. The remaining papers replicate significantly worse than expected (p < 10−14), even when adjusting for regression-to-the-mean of effect size between discovery- and replication-cohorts termed the Winner’s Curse (p < 10−16). We show this is due in part to misreporting replication cohort-size as a maximum number, rather than per-locus one. In 39 studies accurately reporting per-locus cohort-size for attempted replication of 707 loci in samples with similar ancestry, replication rate matched expectation (predicted 458, observed 457, p = 0.94). In contrast, ancestry differences between replication and discovery (13 studies, 385 loci) cause the most highly-powered decile of loci to replicate worse than expected, due to difference in linkage disequilibrium.

Promises and pitfalls of using high‐throughput sequencing for diet analysis

“…The finding of rarer taxon identified in a given diet study will be hard to discover again when replicating the original study or comparing it with another study. This happens because the effect sizes of statistically significant findings are inflated, which has been shown to affect replication of results in many other areas of research (Button et al, 2013;Palmer & Pe'er 2017). Hence, reproducibility and comparability also depend on the taxonomic resolution of the study, as replicating the results at an order level is much more likely than reproducing a species-level finding.…”

Recent Genetics and Epigenetics Approaches to PTSD

“…Additionally, even when successful, these types of results may be difficult to interpret biologically; in particular, it is difficult to translate large lists of associated loci into meaningful mechanisms for follow-up study. Finally, another complication is the effect of “winner’s curse” in GWAS; that is, findings close to the genome-wide significance threshold are likely to be inflated, and consequently do not often replicate [74]. Successful, well-powered GWAS therefore may produce large lists of loci, but these have the risk of being uninformative and overly optimistic.…”

Association of 3p27.1 Variants with Whole Body Lean Mass Identified by a Genome-wide Association Study

“…The identified lead SNP rs3732593 was extremely significant (P = 7.19 × 10 −8 ) in the discovery FHS sample but was only nominally significant (P = 0.04) in the replication sample. Due to the winner's curse effect, replication p-value is difficult to be as low as that in the discovery sample even for true association 33 . Instead, a nominally significant replication p-value would suggest a successful replication.…”