Heterogeneity in Mathematics Intervention Effects: Evidence from a Meta-Analysis of 191 Randomized Experiments

Williams, Ruth; Citkowicz, Martyna; Miller, David I.; Lindsay, Jim; Walters, Kate

doi:10.1080/19345747.2021.2009072

Cited by 13 publications

(17 citation statements)

References 183 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…30 There has been significant research to increase the practical utility of meta-regression, for example implementing machine learning approaches or using cross-validation for model evaluation. 31,32 Other research suggests an information-theoretic approach to improve model selection in meta-analysis models, which is not a focus of this methodological paper but will be of interest to those wishing to use such methods in practice. 33 Robust variance estimation, in which both between-study and within-study variance are properly accounted for, has also been proposed for meta-regression models.…”

Section: Traditional Meta-analysis Methodsmentioning

confidence: 99%

Causally interpretable meta‐analysis: Clearly defined causal effects and two case studies

Rott,

Bronfort,

Chu

et al. 2023

Research Synthesis Methods

View full text Add to dashboard Cite

Meta‐analysis is commonly used to combine results from multiple clinical trials, but traditional meta‐analysis methods do not refer explicitly to a population of individuals to whom the results apply and it is not clear how to use their results to assess a treatment's effect for a population of interest. We describe recently‐introduced causally interpretable meta‐analysis methods and apply their treatment effect estimators to two individual‐participant data sets. These estimators transport estimated treatment effects from studies in the meta‐analysis to a specified target population using the individuals' potentially effect‐modifying covariates. We consider different regression and weighting methods within this approach and compare the results to traditional aggregated‐data meta‐analysis methods. In our applications, certain versions of the causally interpretable methods performed somewhat better than the traditional methods, but the latter generally did well. The causally interpretable methods offer the most promise when covariates modify treatment effects and our results suggest that traditional methods work well when there is little effect heterogeneity. The causally interpretable approach gives meta‐analysis an appealing theoretical framework by relating an estimator directly to a specific population and lays a solid foundation for future developments.

show abstract

Section: Traditional Meta-analysis Methodsmentioning

confidence: 99%

Causally interpretable meta‐analysis: Clearly defined causal effects and two case studies

Rott,

Bronfort,

Chu

et al. 2023

Research Synthesis Methods

View full text Add to dashboard Cite

show abstract

“…If the review includes average impacts across the few studies, it would be appropriate to estimate "fixed effects" (Borenstein, 2019) and the authors should alert the reader to the nature of the few samples, settings, and implementation contexts represented in the analysis. Williams et al (2022) is a rare example of a Meta-Analysis designed from an implicit logic model built on extant theory. The team designed evidence screening and abstraction process that captured data on not only key outcomes and treatment status but also characteristics of the sample, key qualities of the treatments and the settings, and methodological strategies that could affect impact estimates.…”

Section: Improve the Design And Reporting Of Individual Studiesmentioning

confidence: 99%

“… Williams et al (2022) is a rare example of a Meta-Analysis designed from an implicit logic model built on extant theory. The team designed evidence screening and abstraction process that captured data on not only key outcomes and treatment status but also characteristics of the sample, key qualities of the treatments and the settings, and methodological strategies that could affect impact estimates.…”

Section: Room For Improvementmentioning

confidence: 99%

“…The resulting analysis sample included large 191 studies, 1109 effect sizes, and data for more than 250 thousand students. Thus, the study was able to move beyond an estimating the average impact for the pooled sample and systematically explore heterogeneity of impact estimates in ways that informed judgments about generalizability and applicability of findings to particular populations, intervention features and settings ( Williams et al, 2022 , Table 1, page 593). 10…”

Section: Room For Improvementmentioning

confidence: 99%

See 1 more Smart Citation

Improving the Usefulness and Use of Meta-Analysis to Inform Policy and Practice

Maynard

2024

Eval Rev

View full text Add to dashboard Cite

This chapter begins with an overview of recent developments that have encouraged and facilitated greater use of research syntheses, including Meta-Analysis, to guide public policy and practice in education, workforce development, and social services. It discusses the role of Meta-Analysis for improving knowledge of the effectiveness of programs, policies, and practices and the applicability and generalizability of that knowledge to conditions other than those represented by the study samples and settings. The chapter concludes with recommendations for improving the potential of Meta-Analysis to accelerate knowledge development through changing how we design, conduct, and report findings of individual studies to maximize their usefulness in Meta-Analysis as well as how we produce and report Meta-Analysis findings. The paper includes references to resources supporting the recommendations.

show abstract

“…Research Azzolini et al (2022) discussed online courses and their impact on teacher retention. Research by Williams et al (2022) discussed heterogeneity in mathematics learning. Research Karim et al (2022) studied health students' responses to e-learning and focused on the impact on students' mental health with e-learning.…”

Section: Introductionmentioning

confidence: 99%

Exploration of Mindful Learning Systems in Medical Education in the Covid-19 Pandemic Era

Mardhiyah,

Hendarto,

Dwi Susilowati

et al. 2023

IJRP

View full text Add to dashboard Cite

The COVID-19 pandemic has caused changes in various areas of life, including higher education. Medical higher education has a big challenge in the era of the COVID-19 pandemic, because it has limitations in achieving the target of graduates who are experts, both in terms of material and medical practice skills. Consumers must have their needs and desires met through the company's products or services. This study explores students' assessments and expectations regarding a wise learning system, which is following the expectations of students as consumers, in medical education during the COVID-19 pandemic. The findings from exploring the student's assessment and expectations became the basis for forming a wise concept of a medical education system during the COVID-19 pandemic. This is beneficial for educators to find win-win solutions for consumers and medical education providers that are effective and ethical in the era of the COVID-19 pandemic. This qualitative research obtained the data through FGD with medical faculty students as informants and lecturers as a triangulation unit. The academic and non-academic aspects of forming mindful learning will be discussed in more detail. The results of this research can be used if educational institutions are to carry out online distance learning

show abstract

Heterogeneity in Mathematics Intervention Effects: Evidence from a Meta-Analysis of 191 Randomized Experiments

Cited by 13 publications

References 183 publications

Causally interpretable meta‐analysis: Clearly defined causal effects and two case studies

Causally interpretable meta‐analysis: Clearly defined causal effects and two case studies

Improving the Usefulness and Use of Meta-Analysis to Inform Policy and Practice

Exploration of Mindful Learning Systems in Medical Education in the Covid-19 Pandemic Era

Contact Info

Product

Resources

About