Assessing Publication Bias in Meta-Analyses in the Presence of Between-Study Heterogeneity

Peters, Jaime; Sutton, Alex J.; Jones, David R.; Abrams, Keith R.; Rushton, Lesley; Moreno, Santiago

doi:10.1111/j.1467-985x.2009.00629.x

Cited by 130 publications

(124 citation statements)

References 35 publications

Supporting

Mentioning

123

Contrasting

Order By: Relevance

“…This can be formally confirmed by means of a homogeneity test using a commonly used "Q-statistic" (Engels et al 2000). The Q-statistic (computation as in Peters et al 2010) tests if the primary studies share a common effect size and whether an FE estimate is relevant to the analysis (Poot 2014). After combining K effect sizes, if the resulting Q-statistic from this homogeneity test is greater than the upper-tail critical value of the chi-square distribution with K − 1 degrees of freedom, then the variance in effect sizes obtained from the primary studies is significantly greater than what can be observed due to random variation around a common effect size (Shadish, Haddock 1994).…”

Section: Advocacymentioning

confidence: 92%

Infrastructure and Trade: A Meta-Analysis

Celbiş

Nijkamp²,

Poot

2015

REGION

View full text Add to dashboard Cite

Abstract. Low levels of infrastructure quality and quantity can create trade impediments through increased transport costs. Since the late 1990s, an increasing number of trade studies have taken infrastructure into account. The purpose of the present paper is to quantify the importance of infrastructure for trade by means of meta-analysis and meta-regression techniques that synthesize various studies. The type of infrastructure that we focus on is mainly public infrastructure in transportation and communication.We examine the impact of infrastructure on trade by means of estimates obtained from 36 primary studies that yielded 542 infrastructure elasticities of trade. We explicitly take into account that infrastructure can be measured in various ways and that its impact depends on the location of the infrastructure. We estimate several meta-regression models that control for observed heterogeneity in terms of variation across different methodologies, infrastructure types, geographical areas and their economic features, model specifications, and publication characteristics. Additionally, random effects account for between-study unspecified heterogeneity, while publication bias is explicitly addressed by means of the Hedges model. After controlling for these issues, we find that a 1 percent increase in own infrastructure increases exports by about 0.6 percent and imports by about 0.3 percent. Such elasticities are generally larger for developing countries, land infrastructure, IV or panel data estimation, and macro-level analyses. They also depend on the inclusion or exclusion of various common covariates in trade regressions.JEL classification: F10, H54, R53, C10, F1, R4

show abstract

Section: Advocacymentioning

confidence: 92%

Infrastructure and Trade: A Meta-Analysis

Celbiş

Nijkamp²,

Poot

2015

REGION

View full text Add to dashboard Cite

show abstract

“…If the latter is the case, the trim-andfill-adjusted effect size may be underestimating the true effect size (e.g., Peters, Sutton, Jones, Abrams, & Rushton, 2007). Unfortunately, the proposed methods of unconfounding publication bias and moderating factors (e.g., conducting funnel plot analyses within a subgroup of studies) are applicable only to large meta-analyses (see Peters et al, 2010).…”

Section: Stimulant Effects On Healthy People's Delayed Episodic Memorymentioning

confidence: 99%

Prescription Stimulants' Effects on Healthy Inhibitory Control, Working Memory, and Episodic Memory: A Meta-analysis

Ilieva

Hook²,

Farah³

2015

Journal of Cognitive Neuroscience

112

110

View full text Add to dashboard Cite

The use of prescription stimulants to enhance healthy cognition has significant social, ethical, and public health implications. The large number of enhancement users across various ages and occupations emphasizes the importance of examining these drugs' efficacy in a nonclinical sample. The present meta-analysis was conducted to estimate the magnitude of the effects of methylphenidate and amphetamine on cognitive functions central to academic and occupational functioning, including inhibitory control, working memory, short-term episodic memory, and delayed episodic memory. In addition, we examined the evidence for publication bias. Forty-eight studies (total of 1,409 participants) were included in the analyses. We found evidence for small but significant stimulant enhancement effects on inhibitory control and short-term episodic memory. Small effects on working memory reached significance, based on one of our two analytical approaches. Effects on delayed episodic memory were medium in size. However, because the effects on longterm and working memory were qualified by evidence for publication bias, we conclude that the effect of amphetamine and methylphenidate on the examined facets of healthy cognition is probably modest overall. In some situations, a small advantage may be valuable, although it is also possible that healthy users resort to stimulants to enhance their energy and motivation more than their cognition. The large number of enhancement users across various ages and occupations emphasizes the importance of examining these drugs' efficacy in a nonclinical sample. The present meta-analysis was conducted to estimate the magnitude of the effects of methylphenidate and amphetamine on cognitive functions central to academic and occupational functioning, including inhibitory control, working memory, short-term episodic memory, and delayed episodic memory. In addition, we examined the evidence for publication bias. Forty-eight studies (total of 1,409 participants) were included in the analyses. We found evidence for small but significant stimulant enhancement effects on inhibitory control and short-term episodic memory. Small effects on working memory reached significance, based on one of our two analytical approaches. Effects on delayed episodic memory were medium in size. However, because the effects on long-term and working memory were qualified by evidence for publication bias, we conclude that the effect of amphetamine and methylphenidate on the examined facets of healthy cognition is probably modest overall. In some situations, a small advantage may be valuable, although it is also possible that healthy users resort to stimulants to enhance their energy and motivation more than their cognition. ■

show abstract

“…We thus suggest that methods such as the trim and fill can be used along with selection models, cumulative meta-analysis, or other methods under varying conditions in simulation studies. Furthermore, simulation studies could consider whether the nature and/or form of the heterogeneity (Peters et al, 2010) affect publication bias results. Such studies can also explore whether relative indices (e.g., I…”

Section: Limitations and Recommendationsmentioning

confidence: 99%

Publication Bias in the Organizational Sciences

Kepes

Banks

McDaniel

et al. 2012

Organizational Research Methods

355

450

View full text Add to dashboard Cite

Publication bias poses multiple threats to the accuracy of meta-analytically derived effect sizes and related statistics. Unfortunately, a review of the literature indicates that unlike meta-analytic reviews in medicine, research in the organizational sciences tends to pay little attention to this issue. In this article, the authors introduce advances in meta-analytic techniques from the medical and related sciences for a comprehensive assessment and evaluation of publication bias. The authors illustrate their use on a data set on employment interview validities. Using multiple methods, including contour-enhanced funnel plots, trim and fill, Egger’s test of the intercept, Begg and Mazumdar’s rank correlation, meta-regression, cumulative meta-analysis, and selection models, the authors find limited evidence of publication bias in the studied data.

show abstract

Assessing Publication Bias in Meta-Analyses in the Presence of Between-Study Heterogeneity

Cited by 130 publications

References 35 publications

Infrastructure and Trade: A Meta-Analysis

Infrastructure and Trade: A Meta-Analysis

Prescription Stimulants' Effects on Healthy Inhibitory Control, Working Memory, and Episodic Memory: A Meta-analysis

Publication Bias in the Organizational Sciences

Contact Info

Product

Resources

About