The False-positive to False-negative Ratio in Epidemiologic Studies

Abstract

The ratio of false-positive to false-negative findings (FP:FN ratio) is an informative metric that warrants further evaluation. The FP:FN ratio varies greatly across different epidemiologic areas. In genetic epidemiology, it has varied from very high values (possibly even >100:1) for associations reported in candidate-gene studies to very low values (1:100 or lower) for associations with genome-wide significance. The substantial reduction over time in the FP:FN ratio in human genome epidemiology has corresponded to the routine adoption of stringent inferential criteria and comprehensive, agnostic reporting of all analyses. Most traditional fields of epidemiologic research more closely follow the practices of past candidate gene epidemiology, and thus have high FP:FN ratios. Further, FP and FN results do not necessarily entail the same consequences, and their relative importance may vary in different settings. This ultimately has implications for what is the acceptable FP:FN ratio and for how the results of published epidemiologic studies should be presented and interpreted.

When null hypothesis significance testing is unsuitable for research: a reassessment

“…For example, half or more of the drugs tested in large, late phase III trials show higher effectiveness against older comparators (H 0 :H 1 = <1; Soares et al, 2005). Conversely, the vast majority of tested hypotheses in large-scale exploratory research reflect null effects, e.g., in the search of genetic variants associated with various diseases in the candidate gene era where investigators were asking hypotheses one or a few at a time (the same way that investigators continue to test hypotheses in most other biomedical and social science fields) yielded thousands of putative discovered associations, but only 1.2% of them were subsequently validated to be non-null when large-scale consortia with accurate measurements and rigorous analyses plans assessed them (Chanock et al, 2007;Ioannidis et al, 2011). Of the hundreds of thousands to many millions of variables assessed in current agnostic-omics testing, much less than 1% are likely to reflect non-null effects (H 0 :H 1 >>100).…”

Proton Pump Inhibitors and Dementia: Deciphering the Data

“…As such, any clinical diagnosis of AD will be inaccurate and so evaluating specifi c types of dementia outside of post mortem studies will suff er from misclassifi cation bias. Th is is unlikely to be the explanation for the AJG study being negative; however, a previous smaller cohort study (involving 3,327 patients) had found an association between PPI use and patients specifi cally diagnosed with AD, as well as any cause of dementia ( 19 ). Th is is not surprising, as AD accounts for up to 70% of all dementia diagnoses.…”

“…Th is is not surprising, as AD accounts for up to 70% of all dementia diagnoses. Athough this study ( 19 ) reported a statistically signifi cant association, the P -value was modest ( P =0.04 for the association between PPI and AD) and the authors did not adjust for multiple testing. Th e cohort was set up to look for any risk of dementia and not PPIs specifi cally and the authors should therefore have adjusted their analysis for multiple testing.…”

Research on Gene–Environment Interplay in the Era of “Big Data”

“…We have moved beyond the era of small sample candidate gene studies, arguably an era of randomwalk science, with multiple independent teams of investigators pursuing their favorite candidate genes (albeit frequently informed by data from animal models). In hindsight, for addictions as for other complex medical phenotypes, the primary product of that research was a massive accumulation of false-positive publications (Ioannidis et al, 2011)-with rare exceptions, such as genes that influence drug metabolism (e.g., MacGregor et al, 2009).…”