Multilevel meta-analysis of single-case experimental designs using robust variance estimation.

Abstract

Single-case experimental designs (SCEDs) are used to study the effects of interventions on the behavior of individual cases, by making comparisons between repeated measurements of an outcome under different conditions. In research areas where SCEDs are prevalent, there is a need for methods to synthesize results across multiple studies. One approach to synthesis uses a multilevel meta-analysis (MLMA) model to describe the distribution of effect sizes across studies and across cases within studies. However, MLMA relies on having accurate sampling variances of effect size estimates for each case, which may not be possible due to auto-correlation in the raw data series. One possible solution is to combine MLMA with robust variance estimation (RVE), which provides valid assessments of uncertainty even if the sampling variances of effect size estimates are inaccurate. Another possible solution is to forgo MLMA and use simpler, ordinary least squares (OLS) methods with RVE. This study evaluates the performance of effect size estimators and methods of synthesizing SCEDs in the presence of auto-correlation, for several different effect size metrics, via a Monte Carlo simulation designed to emulate the features of real data series. Results demonstrate that the MLMA model with RVE performs properly in terms of bias, accuracy, and confidence interval coverage for estimating overall average log response ratios. The OLS estimator corrected with RVE performs the best in estimating overall average Tau effect sizes. None of the available methods perform adequately for meta-analysis of within-case standardized mean differences.

Effectiveness of Reading Interventions on Word Recognition for Students with Intellectual Disability: A Systematic Review and IPD Meta-analysis of Single Case Experimental Designs

“…With the increased importance of evidence available from SCEDs and proliferation of statistical analyses of SCEDs (cf. Chen & Pustejovsky, 2022;Pustejovsky, 2019;Swan & Pustejovsky, 2018;Odom et al, 2018), it is necessary to promote open and reproducible methods of synthesizing this evidence and assessing the quality of conducted SCED studies.…”

Meta-analysis of single-case design research: Application of multilevel modeling.

“…To address the methodological limitations in quantifying the learners’ behavior changes observed in single-case designs, researchers have suggested the use of advanced statistical approaches using multilevel modeling for single-case data (e.g., Chen & Pustejovsky, 2022; Van den Noortgate & Onghena, 2003a, 2003b). Recent efforts to aggregate data based on raw and individual participant data and the advancement of computing software have allowed practitioners and researchers to apply quantifiable approaches across students and studies (Declercq et al, 2022; Dowdy et al, 2021).…”

DRA dengan Token Ekonomi untuk Menurunkan Perilaku Makan yang Menyimpang: Sebuah Studi Kasus pada Anak

“…Penelitian ini menggunakan desain studi kasus tunggal atau Single Case Experimental Design (SCED). SCED adalah desain penelitian yang digunakan untuk mempelajari efek dari intervensi pada perilaku dalam kasus individu seiring berjalannya waktu (Chen & Pustejovsky, 2022). Karakteristik utama dari SCED adalah adanya pengukuran perilaku yang dilakukan secara berulang pada kondisi yang berbeda, misal sebelum dan sesudah suatu intervensi dilakukan (Kazdin, 2019).…”