A systematic review of single-case experimental design meta-analyses: characteristics of study designs, data, and analyses

Jamshidi, Laleh; Heyvaert, Mieke; Declercq, Lies; Fernández-Castilla, Belén; Ferron, John M.; Moeyaert, Mariola; Beretvas, S. Natasha; Onghena, Patrick; Noortgate, Wim Van Den

doi:10.1080/17489539.2022.2089334

Cited by 19 publications

(23 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these techniques reveal relatively little about the large amount of raw data from which the effect size was computed. In addition, these techniques are not common ly used in meta-analyses (Jamshidi et al, 2022;Natesan, 2019), probably because they are not well understood by practitioners. Specifically, using multilevel models requires making several complex modeling decisions that may have an impact on the validity of the estimates (Baek & Ferron, 2020;Moeyaert et al, 2016).…”

Section: Top-down Versus Bottom-up Meta-analysis Of Scedsmentioning

confidence: 99%

Violin plots as visual tools in the meta-analysis of Single-Case Experimental Designs

Tanious

Manolov

2022

Methodology

View full text Add to dashboard Cite

Despite the existence of sophisticated statistical methods, systematic reviews regularly indicate that single-case experimental designs (SCEDs) are predominantly analyzed through visual tools. For the quantitative aggregation of results, different meta-analytical techniques are available, but specific visual tools for the meta-analysis of SCEDs are lacking. The present article therefore describes the use of violin plots as visual tools to represent the raw data. We first describe the underlying rationale of violin plots and their main characteristics. We then show how the violin plots can complement the statistics obtained in a quantitative meta-analysis. The main advantages of violin plots as visual tools in meta-analysis are (a) that they preserve information about the raw data from each study, (b) that they have the ability to visually represent data from different designs in one graph, and (c) that they enable the comparison of score distributions from different experimental phases from different studies.

show abstract

Section: Top-down Versus Bottom-up Meta-analysis Of Scedsmentioning

confidence: 99%

Violin plots as visual tools in the meta-analysis of Single-Case Experimental Designs

Tanious

Manolov

2022

Methodology

View full text Add to dashboard Cite

show abstract

“…The two-stage approach is relevant when studies use different methods of assessing outcomes—making the one-stage approach infeasible—but where effect size metrics can be calculated and compared across studies. In the present study, we will focus on the two-stage approach because it is comparatively simpler and has been more widely applied (Jamshidi et al, in press).…”

mentioning

confidence: 99%

“…In the present study, we will focus on the two-stage approach because it is comparatively simpler and has been more widely applied (Jamshidi et al, 2020).…”

mentioning

confidence: 99%

Multilevel meta-analysis of single-case experimental designs using robust variance estimation.

Chen¹,

Pustejovsky²

2022

Psychological Methods

View full text Add to dashboard Cite

Single-case experimental designs (SCEDs) are used to study the effects of interventions on the behavior of individual cases, by making comparisons between repeated measurements of an outcome under different conditions. In research areas where SCEDs are prevalent, there is a need for methods to synthesize results across multiple studies. One approach to synthesis uses a multilevel meta-analysis (MLMA) model to describe the distribution of effect sizes across studies and across cases within studies. However, MLMA relies on having accurate sampling variances of effect size estimates for each case, which may not be possible due to auto-correlation in the raw data series. One possible solution is to combine MLMA with robust variance estimation (RVE), which provides valid assessments of uncertainty even if the sampling variances of effect size estimates are inaccurate. Another possible solution is to forgo MLMA and use simpler, ordinary least squares (OLS) methods with RVE. This study evaluates the performance of effect size estimators and methods of synthesizing SCEDs in the presence of auto-correlation, for several different effect size metrics, via a Monte Carlo simulation designed to emulate the features of real data series. Results demonstrate that the MLMA model with RVE performs properly in terms of bias, accuracy, and confidence interval coverage for estimating overall average log response ratios. The OLS estimator corrected with RVE performs the best in estimating overall average Tau effect sizes. None of the available methods perform adequately for meta-analysis of within-case standardized mean differences.

show abstract

“…Some of these six data aspects have received greater attention in the scientific literature than others. Specifically, the data aspect "overlap" has received a great deal of attention, with the Percentage of Nonoverlapping Data (Scruggs et al, 1987) as one of the first and most widely adopted quantification of effect size in SCEDs, and with other more recent nonoverlap indices as the most frequently used alternatives for the quantification of SCED effects (Jamshidi et al, 2022;Maggin, O'Keefe, & Johnson, 2011;Radley et al, 2020).…”

mentioning

confidence: 99%

“…Some of these six data aspects have received greater attention in the scientific literature than others. Specifically, the data aspect “overlap” has received a great deal of attention, with the Percentage of Nonoverlapping Data (Scruggs et al, 1987 ) as one of the first and most widely adopted quantification of effect size in SCEDs, and with other more recent nonoverlap indices as the most frequently used alternatives for the quantification of SCED effects (Jamshidi et al, 2022 ; Maggin, O'Keefe, & Johnson, 2011 ; Radley et al, 2020 ). The different nonoverlap indices have also been compared multiple times (e.g., Lenz, 2013 ; Parker, Vannest, & Davis, 2011 ; Rakap, 2015 ), and included in studies both with real data (e.g., M. Chen et al, 2016 ; Wolery et al, 2010 ) and with generated data (e.g., Giannakakos & Lanovaz, 2019 ; Tarlow, 2017 ).…”

mentioning

confidence: 99%

Defining and assessing immediacy in single‐case experimental designs

Manolov

Onghena

2022

J Exper Analysis Behavior

Self Cite

View full text Add to dashboard Cite

Immediacy is one of six data aspects (alongside level, trend, variability, overlap, and consistency) that has to be accounted for when visually analyzing single‐case data. Given that it is one of the aspects that has received considerably less attention than other data aspects, the current text offers a review of the proposed conceptual definitions of immediacy (i.e., what it refers to) and also of the suggested operational definitions (i.e., how exactly is it assessed and/or quantified). Provided that a variety of conceptual and operational definitions is identified, we propose following a sensitivity analysis using a randomization test for assessing immediate effects in single‐case experimental designs, by identifying when changes were most clear. In such a sensitivity analysis, the immediate effects are tested for multiple possible intervention points and for different possible operational definitions. Robust immediate effects can be detected if the results for the different operational definitions converge.

show abstract

A systematic review of single-case experimental design meta-analyses: characteristics of study designs, data, and analyses

Cited by 19 publications

References 65 publications

Violin plots as visual tools in the meta-analysis of Single-Case Experimental Designs

Violin plots as visual tools in the meta-analysis of Single-Case Experimental Designs

Multilevel meta-analysis of single-case experimental designs using robust variance estimation.

Defining and assessing immediacy in single‐case experimental designs

Contact Info

Product

Resources

About