Empirical assessment of effect of publication bias on meta-analyses

Abstract

Objective To assess the effect of publication bias on the results and conclusions of systematic reviews and meta-analyses. Design Analysis of published meta-analyses by trim and fill method. Studies 48 reviews in Cochrane Database of Systematic Reviews that considered a binary endpoint and contained 10 or more individual studies. Main outcome measures Number of reviews with missing studies and effect on conclusions of meta-analyses. Results The trim and fill fixed effects analysis method estimated that 26 (54%) of reviews had missing studies and in 10 the number missing was significant. The corresponding figures with a random effects model were 23 (48%) and eight. In four cases, statistical inferences regarding the effect of the intervention were changed after the overall estimate for publication bias was adjusted for. Conclusions Publication or related biases were common within the sample of meta-analyses assessed. In most cases these biases did not affect the conclusions. Nevertheless, researchers should check routinely whether conclusions of systematic reviews are robust to possible non-random selection mechanisms.

Effect of psychological stress on blood pressure increase: a meta-analysis of cohort studies

“…The variability of the selected studies was evaluated through a heterogeneity test using models with fixed effects when the test was statistically nonsignificant (p ≥ 0.05) and random effects when the test was statistically significant (p < 0.05) 14 . Begg's and Egger's tests were used to investigate the existence of publication bias 24 .…”

The effectiveness of nurse-led care in people with rheumatoid arthritis: A systematic review

“…Publication bias occurs when trials showing no effects are selectively not published and this can be tested by plotting sample sizes for each study against the effect sizes. This test is not capable of detecting publication bias with less than 10 trials (Sutton et al, 2000) therefore we were unable to assess for publication bias in this review.…”

Dexamethasone versus prednisone for induction therapy in childhood acute lymphoblastic leukemia: a systematic review and meta-analysis

“…20 Publication bias was investigated using a funnel plot in which the standard error of the effect estimate of each study was plotted against the estimate. 21 An asymmetric plot suggested possible publication bias. 17 Results Figure 1 illustrates the flow diagram of trial identification and selection.…”