Meta-Analysis with Robust Variance Estimation: Expanding the Range of Working Models

Pustejovsky, James E.; Tipton, Elizabeth

doi:10.31222/osf.io/vyfcj

Cited by 58 publications

(127 citation statements)

References 25 publications

(34 reference statements)

Supporting

Mentioning

126

Contrasting

Order By: Relevance

“…Consequently, estimations of weights and heterogeneity did not account for possible differences between subgroups, which may have affected the efficiency of the model specifications. 14 A method for combining the two working models has been very recently proposed 25 and may counter this problem in future meta-analyses.…”

Section: Discussionmentioning

confidence: 99%

Computerized Cognitive Training in Cognitively Healthy Older Adults: A Systematic Review and Network Meta-Analysis

Lampit

Gavelin

Sabatés

et al. 2020

Preprint

View full text Add to dashboard Cite

Background Computerized cognitive training (CCT) is a broad category of drill-and-practice interventions aims to maintain cognitive performance in older adults. Despite a supportive evidence base for general efficacy, it is unclear what types of CCT are most likely to be beneficial and what intervention design factors are essential for clinical implementation. Methods We searched MEDLINE, Embase, and PsycINFO to August 2019 for randomized controlled trials (RCTs) of any type of CCT in cognitively healthy older adults. Risk of bias within studies was assessed using the Cochrane Risk of Bias 2 tool. The primary outcome was change in overall cognitive performance between CCT and control groups. Secondary outcomes were individual cognitive domains. A series of meta-regressions were performed to estimates associations between key design factors and overall efficacy using robust variance estimation models. Network meta-analysis was used to compare the main approaches to CCT against passive or common active control conditions. Results Ninety RCTs encompassing 7219 participants across 117 comparisons were included. The overall cognitive effect size across all trials was small (g=0.18, 95% CI 0.14 to 0.23) with considerable heterogeneity (τ2=0.074, 95% prediction interval -0.36 to 0.73), robust to small-study effect or risk of bias. Effect sizes for individual cognitive domains were small, heterogeneous and statistically significant apart from fluid intelligence and visual processing. Meta-regressions revealed significantly larger effect sizes in trials using supervised training or up to three times per week. Multidomain training was the most efficacious CCT approach against any type of control, with greater benefits in a subset of supervised training studies. Conclusions The efficacy of CCT varies substantially across designs, independent of the type of control. Multidomain supervised CCT appears to be the most efficacious approach, and should be developed to accommodate for individual needs and remote delivery settings. Future research should focus on identifying the intervention components and regimens that could attenuate aging-related cognitive decline.

show abstract

Section: Discussionmentioning

confidence: 99%

Computerized Cognitive Training in Cognitively Healthy Older Adults: A Systematic Review and Network Meta-Analysis

Lampit

Gavelin

Sabatés

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, several research groups have conducted multiple studies across different publications. To account for non-independence, a two-stage random effects multivariate meta-analysis was performed (using the "metafor" and "clubSandwich" packages in R; Pustejovsky and Tipton 2021;Viechtbauer 2010). In addition to the multivariate structure, the data had some forms of hierarchical structure (i.e.…”

Section: Systematic Behaviour Observations and Analysis Of Think-aloud Protocolsmentioning

confidence: 99%

“…For this study, we assumed that the groups were entirely independent so that the number of independent studies in the model was 30. Following the approach recommended by Pustejovsky and Tipton (2021), we started analysing data by conducting a random effects multivariate meta-analysis known as subgroup correlated effects. In this model, we included random effects for each outcome within each study and each research group.…”

Section: Systematic Behaviour Observations and Analysis Of Think-aloud Protocolsmentioning

confidence: 99%

A Meta-Analysis Investigating the Association Between Metacognition and Math Performance in Adolescence

Muncer¹,

Higham

Gosling

et al. 2021

Educ Psychol Rev

View full text Add to dashboard Cite

Poor math and numeracy skills are associated with a range of adverse outcomes, including reduced employability and poorer physical and mental health. Research has increasingly focused on understanding factors associated with the improvement of math skills in school. This systematic literature review and meta-analysis investigated the association between metacognition and math performance in adolescence (11–16-year-olds). A systematic search of electronic databases and grey literature (to 04.01.2020) highlighted 31 studies. The quantitative synthesis of 74 effect sizes from 29 of these studies (30 independent populations) indicated a significantly positive correlation between metacognition and math performance in adolescence (r = .37, 95% CI = [.29, .44], p < .001). There was significant heterogeneity between studies. Consideration of online (versus offline) measures of metacognition and more complex (versus simple) measures of math performance, and their combination, was associated with larger effect sizes; however, heterogeneity remained high for all analyses.

show abstract

“…For illustrative purposes and to more closely reproduce the meta-analysts' reported results, we analyzed the dataset using a simple random-effects model estimated by restricted maximum likelihood and with standard errors estimated with the Knapp-Hartung adjustment. However, note that a best-practice meta-analysis would account for the clustering via, for example, robust estimation or multilevel modeling, or a combination (Pustejovsky & Tipton, 2021).…”

Section: Calculating An E-value For the Point Estimate And Its Confidence Intervalmentioning

confidence: 99%

Methods to Address Confounding and Other Biases in Meta-Analyses: Review and Recommendations

Mathur

VanderWeele

2022

Annu. Rev. Public Health

View full text Add to dashboard Cite

Meta-analyses contribute critically to cumulative science, but they can produce misleading conclusions if their constituent primary studies are biased, for example by unmeasured confounding in nonrandomized studies. We provide practical guidance on how meta-analysts can address confounding and other biases that affect studies’ internal validity, focusing primarily on sensitivity analyses that help quantify how biased the meta-analysis estimates might be. We review a number of sensitivity analysis methods to do so, especially recent developments that are straightforward to implement and interpret and that use somewhat less stringent statistical assumptions than do earlier methods. We give recommendations for how these newer methods could be applied in practice and illustrate using a previously published meta-analysis. Sensitivity analyses can provide informative quantitative summaries of evidence strength, and we suggest reporting them routinely in meta-analyses of potentially biased studies. This recommendation in no way diminishes the importance of defining study eligibility criteria that reduce bias and of characterizing studies’ risks of bias qualitatively. Expected final online publication date for the Annual Review of Public Health, Volume 43 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

show abstract

Meta-Analysis with Robust Variance Estimation: Expanding the Range of Working Models

Cited by 58 publications

References 25 publications

Computerized Cognitive Training in Cognitively Healthy Older Adults: A Systematic Review and Network Meta-Analysis

Computerized Cognitive Training in Cognitively Healthy Older Adults: A Systematic Review and Network Meta-Analysis

A Meta-Analysis Investigating the Association Between Metacognition and Math Performance in Adolescence

Methods to Address Confounding and Other Biases in Meta-Analyses: Review and Recommendations

Contact Info

Product

Resources

About