A parsimonious weight function for modeling publication bias.

Abstract

Quantitative research literature is often biased because studies that fail to find a significant effect (or that demonstrate effects in an undesired or unexpected direction) are less likely to be published. This phenomenon, termed publication bias, can cause problems when researchers attempt to synthesize results using meta-analytic methods. Various techniques exist that attempt to estimate and correct meta-analyses for publication bias. However, there is no single method that can (a) account for continuous moderators by including them within the model, (b) allow for substantial data heterogeneity, (c) produce an adjusted mean effect size, (d) include a formal test for publication bias, and (e) allow for correction when only a small number of effects is included in the analysis. This article describes a method that we believe helps fill that gap. The model uses the beta density as a weight function that represents the selection process and provides adjusted parameter estimates that account for publication bias. Use of the beta density allows us to represent selection using fewer parameters than similar models so that the proposed model is suitable for meta-analyses that include relatively few studies. We explain the model and its rationale, illustrate its use with a real data set, and describe the results of a simulation study that shows the model's utility. (PsycINFO Database Record

Methodological Guidance Paper: High-Quality Meta-Analysis in a Systematic Review

“…These methods, however, do not perform well particularly when there is heterogeneity among effect sizes (Macaskill, Walter, & Irwig, 2001;Pustejovsky & Rodgers, 2019). It is increasingly common for meta-analysts to use selection modeling strategies to explore the robustness of meta-analysis to publication bias (Citkowicz & Vevea, 2017;Vevea & Hedges, 1995). Selection models provide meta-analysts with an estimate of the presence of selective reporting bias by explicitly modeling the process by which studies are chosen for inclusion in a publication.…”

Testing for funnel plot asymmetry of standardized mean differences

“…The basic 3PSM has been extended in several ways, including incorporation of covariates to explain variation in average effect sizes, 55 and further extensions continue to be developed and studied. [56][57][58][59] Findings from the simulation study have several limitations that should be noted. Although we have described the modified funnel plot asymmetry tests in general terms that could be applied to effect size estimates from a range of different study designs, the simulations were limited to SMD effect size estimates generated from simple, two-group between-subjects designs, as might be found in brief psychological experiments conducted in laboratory settings.…”

“…Although the 3PSM still remains less widely known than Eggers' regression and other tests of funnel plot asymmetry, recently developed, easy‐to‐use software tools make the technique much more accessible to applied researchers than in the past. The basic 3PSM has been extended in several ways, including incorporation of covariates to explain variation in average effect sizes, and further extensions continue to be developed and studied …”

Results Blind Science Publishing

“…There have been some statistical methods proposed and used to try to correct for publication bias, for example, as regards meta-analysis (Citkowicz & Vevea, 2017), but there seems to be a widespread assumption that nothing can be done about the bias, or worse, that it is a good practice and nothing should be done about it (e.g., see Schwarzkopf, 2015), that is, given limited space in journals, we need to know only about the studies that found something, not the ones that “didn’t find anything.”…”