Methodological Guidance Paper: High-Quality Meta-Analysis in a Systematic Review

Pigott, Therese D.; Polanin, Joshua R.

doi:10.3102/0034654319877153

Cited by 318 publications

(291 citation statements)

References 60 publications

(92 reference statements)

Supporting

Mentioning

256

Contrasting

Unclassified

Order By: Relevance

“…Common methods used in group design research for exploring publication bias include the visual inspection of a funnel plot and Egger's test of funnel plot symmetry (Pigott & Polanin, 2020). Although, these tools have been applied to SCD meta-analyses (e.g., Barton et al, 2017), their methodological appropriateness related to SCD meta-analyses has not yet been fully explored (Pustejovsky & Ferron, 2017).…”

Section: Figurementioning

confidence: 99%

Evaluation of publication bias in response interruption and redirection: A meta‐analysis

Dowdy

Tincani

Schneider

2020

J of App Behav Analysis

View full text Add to dashboard Cite

Publication bias is the disproportionate representation of studies with large effects and statistically significant findings in the published research literature. If publication bias occurs in singlecase research design studies on applied behavior-analytic (ABA) interventions, it can result in inflated estimates of ABA intervention effects. We conducted an empirical evaluation of publication bias on an evidence-based ABA intervention for children diagnosed with autism spectrum disorder, response interruption and redirection (RIRD). We determined effect size estimates for published and unpublished studies using 3 metrics, percentage of nonoverlapping data (PND), Hedges' g, and log response ratios (LRR). Omnibus effect size estimates across all 3 metrics were positive, supporting that RIRD is an effective treatment for reducing problem behavior maintained by nonsocial consequences. We observed larger PND for published compared to unpublished studies, small and nonsignificant differences in LRR for published compared to unpublished studies, and significant differences in Hedges' g for published compared to unpublished studies, with published studies showing slightly larger effect. We found little, if any, difference in methodological quality between published and unpublished studies. While RIRD appears to be an effective intervention for challenging behavior maintained by nonsocial consequences, our results reflect some degree of publication bias present in the RIRD research literature.

show abstract

Section: Figurementioning

confidence: 99%

Evaluation of publication bias in response interruption and redirection: A meta‐analysis

Dowdy

Tincani

Schneider

2020

J of App Behav Analysis

View full text Add to dashboard Cite

show abstract

“…Any study with an answer of "no" was excluded; all others were extracted to a Microsoft Excel spreadsheet. 2 Finally, from this list of primary studies, we sought to identify the primary study's contact author. If the email was available in the primary study, we extracted it.…”

Section: Samplementioning

confidence: 99%

“…Perhaps surprisingly, the upper right-hand quadrant, unregistered-reported, is a common phenomenon in meta-analyses: the study authors did not describe the measure or data collection, yet the outcome appeared in summary tables or in the final datasets published via data repositories. The former case, where data were reported in tables only, is a prime reason for conducting forward and backward reference harvesting [2]. Studies where outcomes were not articulated in the abstract or text, but only in tables, may not be identified in traditional database searches; however, other authors may have included the study in their meta-analysis.…”

Section: Introductionmentioning

confidence: 99%

Locating unregistered and unreported data for use in a social science systematic review and meta-analysis

et al. 2020

Self Cite

View full text Add to dashboard Cite

Meta-analysts rely on the availability of data from previously conducted studies. That is, they rely on primary study authors to register their outcome data, either in a study's text or on publicly available websites, and report the results of their work, either again in a study's text or on publicly accessible data repositories. If a primary study author does not register data collection and similarly does not report the data collection results, the meta-analyst is at risk of failing to include the collected data. The purpose of this study is to attempt to locate one type of metaanalytic data: findings from studies that neither registered nor reported the collected outcome data. To do so, we conducted a large-scale search for potential studies and emailed an author query request to more than 600 primary study authors to ask if they had collected eligible outcome data. We received responses from 75 authors (12.3%), three of whom sent eligible findings. The results of our search confirmed our proof of concept (i.e., that authors collect data but fail to register or report it publicly), and the meta-analytic results indicated that excluding the identified studies would change some of our substantive conclusions. Cost analyses indicated, however, a high price to finding the missing studies. We end by reaffirming our calls for greater adoption of primary study preregistration as well as data archiving in publicly available repositories.

show abstract

“…Within the context of systematic reviews, a meta-analysis is “a set of statistical techniques for synthesizing the results of multiple studies. Such techniques are used when the guiding research question focuses on a quantitative summary of study results” (Pigott and Polanin 2020 , p. 24). Pigott and Polanin ( 2020 ) recommend the following best practices while conducting a meta-analysis, (a) developing a research question for a meta-analysis; (b) searching all eligible studies; (c) unbiased screening of abstracts and full-text; (d) coding important moderators of effect size variability; and (e) computing and reporting all effect sizes.…”

Section: Introductionmentioning

confidence: 99%

A synthesis of systematic review research on emerging learning environments and technologies

Martin

Dennen

Bonk

2020

Education Tech Research Dev

View full text Add to dashboard Cite

In this introduction to the special issue on systematic reviews on emerging learning environments and technologies, we introduce best practices for conducting systematic reviews and meta-analysis and discuss the need for a systematic review on emerging learning environments and technologies. We synthesize research on seven primary areas of emerging learning environments and technologies that include: (1) social media, (2) massive open online courses, (3) special education technology, (4) mobile learning, (5) game-based learning and gamification, (6) adaptive learning, and (7) learning analytics and introduce the thirteen articles that were included in this special issue. This article also provides implications for the readers on using and conducting systematic reviews.

show abstract

Methodological Guidance Paper: High-Quality Meta-Analysis in a Systematic Review

Cited by 318 publications

References 60 publications

Evaluation of publication bias in response interruption and redirection: A meta‐analysis

Evaluation of publication bias in response interruption and redirection: A meta‐analysis

Locating unregistered and unreported data for use in a social science systematic review and meta-analysis

A synthesis of systematic review research on emerging learning environments and technologies

Contact Info

Product

Resources

About