Examining the normality assumption of a design-comparable effect size in single-case designs

Examining the impact of design-comparable effect size on the analysis of single-case design in special education.

“…Articles generally included participants with five or more data points per condition and infrequently featured nonconcurrent designs. These issues, combined with the small number of participants featured in typical D-CES compatible SCD articles ( Mdn = 4), potentially impede the use of SCD to provide evidence of intervention effectiveness in WWC reviews (L. T. Chen et al, 2024).…”

“…SCD may only be incorporated into an intervention's WWC evidence rating if results can be quantified in a manner consistent with group experimental designs. Much like group design effect sizes (e.g., Cohen's d), the D-CES provides an estimate of the standardized mean difference between two phases by dividing the difference in the population means of each phase by the total measurement variance across and within cases (L. T. Chen et al, 2024). This approach produces a summary effect for an SCD study suitable for integration into meta-analyses including group designs (Shadish et al, 2015).…”

“…The lack of research demonstrating the concurrence between the D-CES and visual analysis led Kratochwill et al (2023) to describe WWC's emphasis on the D-CES as "premature" (p. 204). In addition to these issues, the ability of D-CES to approximate variance is compromised in studies with small numbers of measurement instances (e.g., <16) and participants (e.g., <6) under certain conditions (L. T. Chen et al, 2024). Kratochwill et al (2021) also criticized WWC's approach to meta-analysis.…”

“…Data Points per Condition. Due to concerns regarding the influence of measurement occasions on D-CES (L. T. Chen et al, 2024), we indicated whether the minimum number of adjacent treatment comparisons in qualifying designs (e.g., ABAB) included at least five data points per condition-the minimum inclusion criteria for studies included in Odom et al's (2018) evaluation of the D-CES. Articles in which all participants had the appropriate number of data points were coded as "full."…”

“…The number of participants and measurement instances in a study, rather than being determined by the researcher (cf. L. T. Chen et al, 2024) are often related to the limited availability of certain populations (e.g., people with EBD or developmental disabilities) and the demands of applied settings. One reaction to this reality may be the growing acceptance of procedural adaptations with the potential to save time and accommodate small populations, such as nonconcurrent designs (e.g., Slocum et al, 2022).…”