A Priori Justification for Effect Measures in Single-Case Experimental Designs

Abstract

Availability of data and material:No data have been gathered or re-analyzed in the context of the current manuscript. Nevertheless, there is supplementary material that can be consulted in Appendix A (https://osf.io/t96fc/): it includes quotes from several methodological and statistical articles presenting or discussing specific quantitative data analysis approaches.Code availability (software application or custom code): Several freely-available software applications are mentioned in Appendix B (https://osf.io/t96fc/), but the underlying code for creating these application is not been publicly shared in the context of the current manuscript.

A proposal for the assessment of replication of effects in single‐case experimental designs

“…In relation to the way in which the polygon is defined, it is compulsory that applied researchers establish (and report) criteria for a successful intervention prior to gathering the data. This is well-aligned with current recommendations for avoiding confirmation bias (Laraway et al, 2019;Levin et al, 2017;Manolov, Moeyaert, & Fingerhut, 2022). The modified Brinley plot and the superimposed polygon allow for a wider perspective on the effect of the intervention, across A-B comparisons and across participants.…”

“…Deciding what to quantify (e.g., change in level, change in slope, change in variability; immediate or delayed effect) must be related to the type of effect expected. This recommendation is commonly made in the context of randomization tests (Heyvaert & Onghena, 2014;Levin et al, 2017Levin et al, , 2021Michiels et al, 2017), but also in general in terms of SCED data analysis (Manolov, Moeyaert, & Fingerhut, 2022).…”

“…Ideally, the polygon must be defined prior to gathering or inspecting the data (mean levels in case of the modified Brinley plot), in accordance with the principles of preregistration and transparency (Ariens et al, 2020;Johnson & Cook, 2019;Porcino et al, 2020) and the importance of a priori decisions in data analysis (Manolov, Moeyaert, & Fingerhut, 2022). However, an exception could take place in situations in which it might be required to know the Phase A level first.…”

Verbal interaction pattern analysis in clinical psychology

“…There is a long tradition of idiographic research in clinical psychology ( Kazdin, 1978 ; Hayes, 1981 ; Iwata et al, 1994 ), but the emphasis on this type of research is renewed (e.g., Molenaar, 2004 ; Beltz et al, 2016 ; Piccirillo et al, 2019 ; Kazdin, 2021 ). New professional standards for single-case methods ( Tate et al, 2016 ; Horner and Ferron, 2022 ) and the development of new methods for the interpretation of these designs ( Hedges et al, 2013 ; Kratochwill and Levin, 2014 ; Pustejovsky, 2018 ; Manolov et al, 2021 ) could help us to analyze the learning processes, that occurs at the individual level and are settled in motion by the psychologists’ procedures, that account for the clients change.…”

Performance criteria‐based effect size (PCES) measurement of single‐case experimental designs: A real‐world data study

“…These methods include randomization tests (e.g., Edgington & Onghena, 2007;Onghena & Edgington, 1994), multilevel modeling (e.g., Moeyaert et al, 2014;Van den Noortgate & Onghena, 2003a;2003b, see Becraft et al, 2020 for a helpful tutorial), mixed-effects modeling as an extension of multilevel modeling (e.g., DeHart & Kaplan, 2019), other regression-based models (e.g., Allison & Gorman, 1993;Gorsuch, 1983), interrupted time series analysis (e.g., Gottman & Glass, 1978), and nonoverlap-based methods (e.g., Scruggs et al, 1987;. Currently, the following features are usually sought in the statistical methods for the analysis of SCEDs: (a) use of all data (e.g., Becraft et al, 2020;DeHart & Kaplan, 2019;Maggin et al, 2011;Swan & Pustejovsky, 2018), (b) compatibility with visual analysis (Manolov, Tanious, & Onghena, 2022;Moeyaert et al, 2022), (c) not overly influenced by outliers (e.g., Maggin et al, 2011;Pustejovsky, 2018), (d) account for baseline trend, level change, slope change, variability, and autocorrelation (e.g., Barnard-Brak et al, 2020;Batley & Hedges, 2021;Maggin et al, 2011;Manolov & Moeyaert, 2017;Manolov, Moeyaert, & Fingerhut, 2022), (e) an indicator for immediate effect (e.g., , (f ) capable of capturing the full magnitude of the effect (Maggin et al, 2011;Manolov, Moeyaert, & Fingerhut, 2022), (g) the ability to produce individual-specific and also generalizable values (e.g., Barnard-Brak et al, 2020;Becraft et al, 2020;DeHart & Kaplan, 2019;Moeyaert et al, 2022), (h) easily calculable and interpretable by researchers from a range of disciplines and background...…”