The "File Drawer Problem" of Non-Significant Results: Does It Apply to Biological Research?

Csada, Ryan D.; James, P; Espie, Richard H. M.

doi:10.2307/3546355

Cited by 171 publications

(127 citation statements)

References 10 publications

(13 reference statements)

Supporting

Mentioning

121

Contrasting

Unclassified

Order By: Relevance

“…Like biased reporting of significant effects subject to sampling variability (publication bias), p-hacking leads the published literature to overestimate true effect sizes. Given evidence of publication bias within biology [13], it is plausible that p-hacking also occurs. Trim-and-fill and PET-PEESE cannot adjust adequately for p-hacking [19].…”

Section: Revisiting Van Dongen and Gangestad (2011) With New Meta-anamentioning

confidence: 99%

“…The average correlation between FA and these outcomes (weighted by standard errors) was r = 0.18. However, scholars argue that publication bias leads to an overestimation of population effect sizes in any scientific discipline with incentive structures that heavily favor statistically significant results [12]-and this includes biology [13]. VD&G thus applied multiple procedures to correct for publication bias: trim-and-fill [14], estimated effect size at large sample size (150) (for a related procedure, see [15] on PET-PEESE), and estimated effect size for all studies with N > 150.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

p-Curve and Selection Methods as Meta-Analytic Supplements for Biologists: A Demonstration of Effect Size Estimation in Studies of Human Fluctuating Asymmetry

2017

View full text Add to dashboard Cite

Abstract:Fluctuating asymmetry is hypothesized to predict developmental instability (DI) and fitness outcomes. While published studies largely support this prediction, publication bias remains an issue. Biologists have increasingly turned to meta-analysis to estimate true support for an effect. Van Dongen and Gangestad (VD&G) performed a meta-analysis on studies of fluctuating asymmetry (FA) and fitness-related qualities in humans. They found an average robust effect size, but estimates varied widely. Recently, psychologists have identified limitations in traditional meta-analyses and popular companion adjustments, and have advocated for alternative meta-analytic techniques. P-curve estimates true mean effects using significant published effects; it also detects the presence of p-hacking (where researchers exploit researcher "degrees of freedom"), not just publication bias. Alternative selection methods also provide a means to estimate average effect size correcting for publication bias, but may better account for heterogeneity in effect sizes and publication decisions than p-curve. We provide a demonstration by performing p-curve and selection method analyses on the set of effects from VD&G. We estimate an overall effect size range (r = 0.08-0.15) comparable to VD&G, but with notable differences between domains and techniques. Results from alternative estimation methods can provide corroborating evidence for, as well as insights beyond, traditional meta-analytic estimates.

show abstract

Section: Revisiting Van Dongen and Gangestad (2011) With New Meta-anamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

p-Curve and Selection Methods as Meta-Analytic Supplements for Biologists: A Demonstration of Effect Size Estimation in Studies of Human Fluctuating Asymmetry

2017

View full text Add to dashboard Cite

show abstract

“…see Lake 2000, Huggett 2005, Bestelmeyer 2006, Groffman et al 2006, and it is surprising to us how little cross-citation occurs among different disciplines. There is also the unsettling possibility that thresholds, while widely reported, are not the norm in ecological systems because negative results are rarely reported (Rosenthal 1979, Csada et al 1996.…”

Section: Experimental Studiesmentioning

confidence: 99%

Multiple stable states and relationship between thresholds in processes and states

Petraitis

Hoffman²

2010

Mar. Ecol. Prog. Ser.

View full text Add to dashboard Cite

The concept of thresholds is applied broadly in ecology to both processes and states that exhibit step-like behavior. Thresholds are observed in parameters, equilibrium states, and in states over time, but presence or absence of thresholds at any of these levels does not provide information about the occurrence of thresholds at the other levels. Here we explore the relationship between thresholds and theory of multiple stable states. We present a 2-species Lotka-Volterra model of competition to illustrate that thresholds and hysteresis-like behavior are possible in linear systems. A grazing model is presented to show that multiple stable states are possible without thresholds in the underlying processes. The concept of thresholds within the context of multiple stable states is reviewed in an attempt to resolve some of the confusion that stems from the different meanings of thresholds.

show abstract

“…There even are guidelines available to help such meta-analyses avoid or minimize certain limitations inherent in them, such as biases that arise because indexing services favor journals published in English or because journals prefer to publish positive results (Csada et al, 1996;Møller and Jennions, 2001;van Leeuwen et al, 2001;Rothstein et al, 2005;AleixandreBenavent et al, 2007).…”

Section: Introductionmentioning

confidence: 99%

Meta-análisis de la historia del estudio de las tortugas continentales de Colombia

Bock¹,

Páez²

2017

Acta biol. Colomb.

View full text Add to dashboard Cite

The citations contained in a recent literature review on the freshwater turtle and tortoise species of Colombia were analyzed. Publishing rates on these species have been increasing exponentially since the 1950s, although many of the publications would not be detected using internet-based literature searches. The most common topics of publications were systematics and phylogeography, nesting ecology, and management. In the 1970s and 1980s, the rate of publishing on the ecology and genetics of Colombian populations lagged behind the rate of publishing on these topics for populations of Colombian freshwater turtle and tortoise species located in other countries, but currently the production of the two regions is comparable. Also, in other countries there were trends towards publishing more over time in English and in journals with higher impact factors, but these trends were not significant for publications on Colombian populations. In Colombia, a disproportionate number of studies have been published on large-bodied species that face conservation problems. We argue that future studies of already well-studied Colombian species should focus on evaluating the effectiveness of management programs, and priority also should be given to increasing our knowledge of the many poorly-studied species in the country, especially those currently classified by the IUCN as data deficient (DD).Keywords: knowledge, literature, meta-analysis, publications, Testudines. RESUMENAnalizamos las referencias incluidas en una reciente revisión literaria sobre el estado de conocimiento de las tortugas continentales de Colombia. La tasa de publicaciones sobre estas especies ha incrementado exponencialmente desde los años 1950s, aunque muchas de éstas no podrían haber sido detectadas usando búsquedas de literatura por internet. Los tópicos más comunes de las publicaciones fueron sistemática y filo-geografía, ecología de anidación, y manejo. En los años 1970s y 1980s, la tasa de publicaciones sobre temas de ecología y genética de poblaciones ocupando el territorio colombiano, estuvo por debajo de publicaciones sobre estos dos tópicos con poblaciones de las mismas especies en otras regiones de Suramérica. Sin embargo, actualmente la producción en ambas regiones es comparable. En otros países, detectamos una tendencia temporal significativa de publicar cada vez más en inglés y en revistas de mayor impacto; esta tendencia no fue significativa para las publicaciones sobre poblaciones de tortugas colombianas. En Colombia, un número desproporcionado de las publicaciones son sobre especies de tamaños grandes que enfrentan problemas de conservación. Argumentamos que estudios futuros de las especies de tortugas continentales de Colombia relativamente bien conocidas, se deberían enfocar en evaluar la efectividad de los programas de manejo, y también debemos dar prioridad a investigaciones que aumenten el estado de conocimiento de las especies de tortugas continentales poco estudiadas en el país, especialmente de aquellas que actualmente están cat...

show abstract

The "File Drawer Problem" of Non-Significant Results: Does It Apply to Biological Research?

Cited by 171 publications

References 10 publications

p-Curve and Selection Methods as Meta-Analytic Supplements for Biologists: A Demonstration of Effect Size Estimation in Studies of Human Fluctuating Asymmetry

p-Curve and Selection Methods as Meta-Analytic Supplements for Biologists: A Demonstration of Effect Size Estimation in Studies of Human Fluctuating Asymmetry

Multiple stable states and relationship between thresholds in processes and states

Meta-análisis de la historia del estudio de las tortugas continentales de Colombia

Contact Info

Product

Resources

About