Conducting a meta-analysis in the age of open science: Tools, tips, and practical recommendations.

Moreau, David; Gamble, Beau

doi:10.1037/met0000351

Cited by 62 publications

(63 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, standards of research quality, transparency, and reproducibility-related practices of meta-analyses need to be high. Tools to help researchers carry out a meta-analysis with the best open practices are available (e.g., Lakens et al, 2016;Moreau & Gamble, 2020), as well as a recent update of the PRISMA statement (Page et al, 2020). We also provide some recommendations in Table 2 which are particularly relevant to researchers carrying out evidence synthesis in the field of clinical psychology.…”

Section: Resultsmentioning

confidence: 99%

“…R (R Core Team, 2019) is a free and open software environment and programming language that, along with RStudio, facilitates the production of easily-shared analysis scripts. Moreau and Gamble (2020) share a very useful script template for carrying out a meta-analysis with R using the metafor (Viechtbauer, 2010) package in their OSF project: https://osf.io/5nk92/.…”

Section: Discussionmentioning

confidence: 99%

“…pre-registration, sharing open material and data, and sharing open analysis scripts offer several benefits (Federer et al, 2018;Hardwicke & Ioannidis, 2018b;Nosek & Lindsay, 2018;Nelson et al, 2018;Nosek et al, 2015;Nosek et al, 2019;Popkin, 2019). The importance of promoting and adopting open science practices in meta-analysis has been increasingly recognised in recent years (Lakens et al, 2016;Moreau & Gamble, 2020;Pigott & Polanin, 2020).…”

Section: Effectiveness Of Psychological Interventionsmentioning

confidence: 99%

See 2 more Smart Citations

A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000-2020).

López-Nicolás¹,

Ja²,

Mr³

et al. 2021

Preprint

View full text Add to dashboard Cite

Meta-analysis is a powerful and important tool to synthesize the literature about a research topic. Like other kinds of research, meta-analyses must be reproducible to be compliant with the principles of the scientific method. Furthermore, reproducible meta-analyses can be easily updated with new data and reanalysed applying new and more refined analysis techniques. We attempted to empirically assess the prevalence of transparency and reproducibility-related reporting practices in published meta-analyses from clinical psychology by examining a random sample of 100 meta-analyses. Our purpose was to identify the key points that could be improved with the aim to provide some recommendations to carry out reproducible meta-analyses. We conducted a meta-review of meta-analyses of psychological interventions published between 2000 and 2019. We searched PubMed, PsycInfo and Web of Science databases. A structured coding form to assess transparency indicators was created based on previous studies and existing meta-analysis guidelines. We found major issues concerning: completely reproducible search procedures report, specification of the exact method to compute effect sizes, choice of weighting factors and estimators, lack of availability of the raw statistics used to compute the effect size and of interoperability of available data, and practically total absence of analysis script code sharing. Based on our findings, we conclude with recommendations intended to improve the transparency, openness and reproducibility-related reporting practices of meta-analyses in clinical psychology and related areas.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Effectiveness Of Psychological Interventionsmentioning

confidence: 99%

See 1 more Smart Citation

A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000-2020).

López-Nicolás¹,

Ja²,

Mr³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Following recommendations to minimize bias and provide evidence of a priori analyses intentions [17], the study was preregistered under the international prospective register of systematic reviews (PROSPERO; registration number: CRD42020186882) and on the Open Science Framework (OSF; https://osf.io/x2483/). Moreover, all materials and data are available on the OSF page of the project in order to facilitate reproducibility and transparency of this review [18].…”

Section: Methodsmentioning

confidence: 99%

Does gamification improve physical activity? A systematic review and meta-analysis

Mazéas¹,

Duclos²,

Pereira³

et al. 2020

Preprint

View full text Add to dashboard Cite

Background. The use of gamification to change behaviors and promote physical activity is a promising avenue to tackle the global physical inactivity pandemic and the current prevalence of chronic diseases. However, we do not have yet evidence of the effectiveness of gamified interventions with the existence of mixed results in the literature.Objective. This systematic review and meta-analysis aimed to evaluate the effectiveness of gamified interventions and their healthcare potential by testing the scalability and sustainability of their influence on physical activity (PA) and sedentary behaviors (SB).Methods. Five electronic databases (Pubmed, EMBASE, Scopus, Web of Science and CENTRAL) were searched for randomized controlled trials (RCTs) published since 2010. Eligibility criteria were based on components of PICO (participants, interventions, comparators, and outcomes). Studies were included when they used gamified interventions in daily life with an active or inactive control group, and when they assessed a PA or SB outcome. Studies were excluded when the intervention was based on an active video game using a gaming console. We conducted meta-analyses using a random effects model approach. Sensitivity analyses, influence analyses and publication bias analyses were performed to examine the robustness of our results.Results. The main meta-analysis performed on 16 studies and 2407 participants, revealed a small to medium summary effect of gamified interventions on PA behavior (g = 0.42 (95%CI [0.14, 0.69]). No statistical difference was found between different subgroups (adults vs adolescents, healthy participants vs adults with chronic diseases) and no interaction effects with moderators like age, gender or BMI, suggesting good scalability of gamified interventions. Moreover, the effect appears better on the step count (MD = +1609.56 steps per day (95%CI [372.39, 2846.73]) than on moderate-to-vigorous PA (g = 0.31 (95%CI [-0.19, 0.80]). The long-term effect (measured with follow-up averaging 14 weeks after the end of the intervention) is more volatile with a very small to small effect (g = 0.15, [0.07, 0.23]). Conclusions. This meta-analysis confirms that gamified interventions are promising to promote PA in various publics. However, if the effect persists after the end of the program, suggesting that it is not just a novelty effect due to the playful nature of gamification, it seems volatile with a small long-term effect. The integration of gamification in more global healthcare interventions could be a way to address this low sustainability. Future rigorous trials are needed to explore these perspectives.

show abstract

“…Third, open data (e.g., metadata about each included article, descriptive statistics, notes, and additional information for each study) can provide the basis for further reviews and meta-analyses and increases the transparency about the meta-analytical findings, and again facilitates reproducibility. The tutorial by Moreau and Gamble (2020) serves as an excellent starting point for meta-analysts aiming to conduct transparent, robust, and reproducible research by embracing open science values. The tutorial starts with an introduction to open science more generally and then focuses on specific recommendations regarding meta-analyses (e.g., a comparison of two platforms (Open Science Framework and PROSPERO) for preregistrations taking into account aspects important for meta-analysts.…”

Section: How Tomentioning

confidence: 99%

Central questions about meta-analyses in psychological research: An annotated reading list

2021

View full text Add to dashboard Cite

A large amount of literature is available for researchers who are interested in performing meta-analyses in psychology. However, due to a large number of available sources and meta-analytic approaches, it can be difficult to get started with a meta-analysis when prior experiences are limited. In this annotated reading list, we provide an overview of and comment on 12 recommended sources that address the most relevant questions for conducting and presenting meta-analyses in psychology. Additionally, we point to various further readings and software packages that address more specific meta-analytic topics. With this guide, we aim to provide a starting point for researchers who wish to conduct a meta-analysis and for reviewers and editors who evaluate the quality of manuscripts presenting meta-analytic findings.

show abstract

Conducting a meta-analysis in the age of open science: Tools, tips, and practical recommendations.

Cited by 62 publications

References 35 publications

A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000-2020).

A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000-2020).

Does gamification improve physical activity? A systematic review and meta-analysis

Central questions about meta-analyses in psychological research: An annotated reading list

Contact Info

Product

Resources

About