Single-case design meta-analyses in education and psychology: a systematic review of methodology

Abstract

Meta-analysis is of increasing importance as this quantitative synthesis technique has the potential to summarize a tremendous amount of research evidence, which can help making evidence-based decisions in policy, practice, and theory. This paper examines the single-case meta-analyses within the Education and Psychology fields. The amount of methodological studies related to the meta-analysis of Single-Case Experimental Designs (SCEDs) is increasing rapidly, especially in these fields. This underscores the necessity of a succinct summary to help methodologists identify areas for further development in Education and Psychology research. It also aids applied researchers and research synthesists in discerning when to use meta-analytic techniques for SCED studies based on criteria such as bias, mean squared error, 95% confidence intervals, Type I error rates, and statistical power. Based on the summary of empirical evidence from 18 reports identified through a systematic search procedure, information related to meta-analytic techniques, data generation and analysis models, design conditions, statistical properties, conditions under which the meta-analytic technique is appropriate, and the study purpose(s) were extracted. The results indicate that three-level hierarchical linear modeling is the most empirically validated SCED meta-analytic technique, and parameter bias is the most prominent statistical property investigated. A large number of primary studies (more than 30) and at least 20 measurement occasions per participant are recommended for usage of SCED meta-analysis in Education and Psychology fields.

Data_Sheet_1.PDF

“…Hedge's g was estimated to provide a measure for the effect of the intervention phases on the performance (Shadish et al, 2014). To test for intervention effects and their moderation through ADHD symptoms, hierarchical piecewise regression models (HPLM) across all students were employed (Moeyaert et al, 2023). In these models, the math performance in the progression-based assessment is predicted by the phase and the measurement time point as well as additional variables such as ADHD symptoms.…”

“…HPLM have the advantage that interaction effects can be added to investigate the effect of additional variables on the effectiveness. To account for individual differences, random effects were added to the HPLM (Moeyaert et al, 2023). Analysis strategy regarding acquisition and maintenance was parallelized: General intervention effects were addressed first, before moderation effects of overall ADHD symptoms (ADHD), inattention (IN) and hyperactivity/impulsivity (HI) were added.…”