Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study

Abstract

BackgroundIn meta-analysis, the presence of funnel plot asymmetry is attributed to publication or other small-study effects, which causes larger effects to be observed in the smaller studies. This issue potentially mean inappropriate conclusions are drawn from a meta-analysis. If meta-analysis is to be used to inform decision-making, a reliable way to adjust pooled estimates for potential funnel plot asymmetry is required.MethodsA comprehensive simulation study is presented to assess the performance of different adjustment methods including the novel application of several regression-based methods (which are commonly applied to detect publication bias rather than adjust for it) and the popular Trim & Fill algorithm. Meta-analyses with binary outcomes, analysed on the log odds ratio scale, were simulated by considering scenarios with and without i) publication bias and; ii) heterogeneity. Publication bias was induced through two underlying mechanisms assuming the probability of publication depends on i) the study effect size; or ii) the p-value.ResultsThe performance of all methods tended to worsen as unexplained heterogeneity increased and the number of studies in the meta-analysis decreased. Applying the methods conditional on an initial test for the presence of funnel plot asymmetry generally provided poorer performance than the unconditional use of the adjustment method. Several of the regression based methods consistently outperformed the Trim & Fill estimators.ConclusionRegression-based adjustments for publication bias and other small study effects are easy to conduct and outperformed more established methods over a wide range of simulation scenarios.

Prospective risk of stillbirth and neonatal complications in twin pregnancies: systematic review and meta-analysis

“…Peter’s test was used to evaluate funnel asymmetry by fitting a weighted linear regression with the logit of event rate as dependent variable and the inverse of sample size as independent variable. We computed weights according to the number of events and no events 27. We used a continuity correction for studies with no events by adding 0.5 to the events count and 1 to the total sample size.…”

Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis

“…Moreover, we also tested the funnel plot statistically for asymmetry by regressing the individual effect sizes on the inverse of their respective sample sizes (cf. Moreno et al, 2009;Peters, Sutton, Jones, Abrams, & Rushton, 2006). A significant effect would indicate funnel plot asymmetry and, thus, a potential publication bias.…”

Systematic review and meta-analysis of mortality and digoxin use in atrial fibrillation

“…We assessed publication bias subjectively by visual inspection of Begg's funnel plot [10] and objectively by Egger's regression asymmetry test as funnel plots may be inaccurate in the assessment of very large studies [11,12]. To address the possibility that "N" number of studies possibly were missing from our analysis and these studies, if included in the analysis, would shift the effect size towards the null, we used Orwin's fail-safe N formula.…”