A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis

Duval, Sue; Tweedie, Richard L.

doi:10.1080/01621459.2000.10473905

Cited by 1,659 publications

(1,798 citation statements)

References 18 publications

Supporting

Mentioning

1,729

Contrasting

Unclassified

Order By: Relevance

“…An often used procedure is the trim and fill method (Duval & Tweedie, 2000a, 2000b. However, the trim and fill method does not perform well with heterogeneous meta-analyses (Moreno et al, 2009;Terrin, Schmid, Lau, & Olkin, 2003) and its performance also depends strongly on assumptions about why studies are missing (Borenstein et al, 2009).…”

Section: Discussionmentioning

confidence: 99%

“…A weakness of statistical procedures that test for publication bias, such as the rank correlation test (Begg & Mazumdar, 1994), Egger's test (Egger, Davey Smith, Schneider, & Minder, 1997), the trim and fill method (Duval & Tweedie, 2000a, 2000b, or Ioannidis and Trikalinos' test for an excess of significant findings (Ioannidis & Trikalinos, 2007; for an extensive discussion about this test and its usage see, e.g., Ioannidis, 2013;Morey, 2013;Simonshon, 2013;Vandekerckhove, Guan, & Styrcula, 2013), is that their statistical power is usually low for meta-analyses with a typical number of studies.…”

Section: Discussionmentioning

confidence: 99%

“…However, the power of these methods is generally low, so the absence of significant evidence for publication bias does not necessarily mean that there is no publication bias (see also Chapter 7). There are also procedures aimed at estimating effect sizes that are either robust against publication bias or correct for it (Copas, 2013;Duval & Tweedie, 2000a, 2000bHedges, 1984;Hedges & Vevea, 1996Simonsohn et al, 2014;Stanley & Doucouliagos, 2014;van Assen et al, 2015;Vevea & Hedges, 1995;Vevea & Woods, 2005). Note that these methods are not without criticism.…”

Section: Solutionsmentioning

confidence: 99%

See 2 more Smart Citations

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Nuijten¹

2018

Preprint

View full text Add to dashboard Cite

SUMMARY DOCTORAL DISSERTATION: Psychology is facing a “replication crisis”. Many psychological findings could not be replicated in novel samples, which lead to the growing concern that many published findings are overly optimistic or even false. In this dissertation, we investigated potential indicators of problems in the published psychological literature. In Part I of this dissertation, we looked at inconsistencies in reported statistical results in published psychology papers. To facilitate our research, we developed the free tool statcheck; a “spellchecker” for statistics. In Part II, we investigated bias in published effect sizes. We showed that in the presence of publication bias, the overestimation of effects can become worse if you combine studies. Indeed, in meta-analyses from the social sciences we found strong evidence that published effects are overestimated. These are worrying findings, and it is important to think about concrete solutions to improve the quality of psychological research. Some of the solutions we propose are preregistration, replication, and transparency. We argue that to select the best strategies to improve psychological science, we need research on research: meta-research.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Solutionsmentioning

confidence: 99%

See 1 more Smart Citation

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Nuijten¹

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Throughout the years a number of techniques have been developed for assessing whether data is missing in a meta-analytic research project and, if so, how this affects the results. Examples of well-known techniques are the Rosenthal's fail-safe test (1979), Egger's linear regression test (Egger, Davey-Smith, Schneider, & Minder, 1997), the Begg and Mazumdar's Rank Correlation test (Begg & Mazumdar, 1994), and the trim-and-fill method (Duval & Tweedie, 2000a, 2000b. It is good practice for a research synthesist to discuss the extent to which the results were affected by missing data, and to…”

Section: Missing Data and Size Of The Data Setmentioning

confidence: 99%

Fitting three-level meta-analytic models in R: A step-by-step tutorial

Assink¹,

Wibbelink²

2016

TQMP

733

846

View full text Add to dashboard Cite

Applying a multilevel approach to meta-analysis is a strong method for dealing with dependency of effect sizes. However, this method is relatively unknown among researchers and, to date, has not been widely used in meta-analytic research. Therefore, the purpose of this tutorial was to show how a three-level random effects model can be applied to meta-analytic models in R using the rma.mv function of the metafor package. This application is illustrated by taking the reader through a step-by-step guide to the multilevel analyses comprising the steps of (1) organizing a data file; (2) setting up the R environment; (3) calculating an overall effect; (4) examining heterogeneity of within-study variance and between-study variance; (5) performing categorical and continuous moderator analyses; and (6) examining a multiple moderator model. By example, the authors demonstrate how the multilevel approach can be applied to meta-analytically examining the association between mental health disorders of juveniles and juvenile offender recidivism. In our opinion, the rma.mv function of the metafor package provides an easy and flexible way of applying a multi-level structure to meta-analytic models in R. Further, the multilevel meta-analytic models can be easily extended so that the potential moderating influence of variables can be examined.

show abstract

“…As a result of these limitations, moderator effects must be interpreted cautiously. Fifth, this review included only peer-reviewed journal articles involving single-case research, which may weaken the validity of the findings (Duval and Tweedie 2000). By excluding group studies and unpublished studies, it does not synthesize all available evidence on the effects of interventions developed to improve functional living skills of individuals with ASD.…”

Section: Limitationsmentioning

confidence: 99%

Meta-analysis of Single-Case Research on Teaching Functional Living Skills to Individuals with ASD

Ninci

Neely

Hong

et al. 2015

Rev J Autism Dev Disord

View full text Add to dashboard Cite

A meta-analysis of 52 studies teaching functional living skills to individuals with autism spectrum disorder was conducted. Using the Tau effect size with the Dunn and the KruskalWallis post-hoc analyses, the following categories were analyzed: age, diagnosis, intervention type, dependent variable, setting, and implementer. Analyses for age yielded statistically significant findings supporting greatest outcomes for elementaryaged individuals compared to secondary-aged individuals as well as adolescents and adults in comparison with preschool-and secondary-aged individuals. Moderate to strong effect sizes were noted across categories for diagnosis, intervention, and dependent variable. Outcomes indicated strong effects across categories for setting and implementer. Convergent validity of Tau effect sizes with visually analyzed ratings of evidence was evaluated, which largely resulted in correspondence.

show abstract

A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis

Cited by 1,659 publications

References 18 publications

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Fitting three-level meta-analytic models in R: A step-by-step tutorial

Meta-analysis of Single-Case Research on Teaching Functional Living Skills to Individuals with ASD

Contact Info

Product

Resources

About