Meta‐analyses and effect sizes in applied behavior analysis: A review and discussion

Abstract

For more than four decades, researchers have used meta-analyses to synthesize data from multiple experimental studies often to draw conclusions that are not supported by individual studies. More recently, single-case experimental design (SCED) researchers have adopted meta-analysis techniques to answer research questions with data gleaned from SCED experiments. Metaanalyses enable researchers to answer questions regarding intervention efficacy, generality, and condition boundaries. Here we discuss meta-analysis techniques, the rationale for their adaptation with SCED studies, and current indices used to quantify the effect of SCED data in applied behavior analysis.

A proposal for the assessment of replication of effects in single‐case experimental designs

“…Visual inspection has a long history in the SCED context (e.g., Miller, 1985;Parker et al, 2006) and is considered important and necessary even with the current abundance of statistical techniques (DeRosa et al, 2021;Ferron et al, 2017;Kipfmiller et al, 2019;Maggin et al, 2018;Ninci, 2019;Wolfe et al, 2019). This is consistent with the training received by certain professionals (Wolfe & McCammon, 2022), with applied researchers' priorities when analyzing data (Byiers et al, 2021), and within actual practice (Dowdy et al, 2021). Moreover, visually inspecting data has been emphasized as indispensable when performing classical statistical analyses outside the SCED context (Fife et al, 2021).…”

Performance criteria‐based effect size (PCES) measurement of single‐case experimental designs: A real‐world data study

“…k FS is a metric that gives information about when the intervention phase can be terminated or shifted to other (Barnard‐Brak et al, 2018). k FS indicates whether the data collected is clinically significant and optimal (Dowdy et al, 2021). We refer to readers interested in k FS to Barnard‐Brak et al (2018).…”

“…Numerous statistical methods for SCEDs have been proposed, to date (Dowdy et al, 2021; Manolov & Moeyaert, 2017). These methods include randomization tests (e.g., Edgington & Onghena, 2007; Onghena & Edgington, 1994), multilevel modeling (e.g., Moeyaert et al, 2014; Van den Noortgate & Onghena, 2003a; 2003b, see Becraft et al, 2020 for a helpful tutorial), mixed‐effects modeling as an extension of multilevel modeling (e.g., DeHart & Kaplan, 2019), other regression‐based models (e.g., Allison & Gorman, 1993; Gorsuch, 1983), interrupted time series analysis (e.g., Gottman & Glass, 1978), and nonoverlap‐based methods (e.g., Scruggs et al, 1987; Parker & Vannest, 2009; Parker, Vannest & Davis, 2011).…”

“…Out of the six data aspects, the most commonly used quantitative methods in the statistical analysis of SCEDs are nonoverlap‐based effect size methods (Dowdy et al, 2021; Jamshidi et al, 2018; Maggin et al, 2011; Tanious et al, 2020). Nonoverlap‐based measures are typically free of statistical assumptions such as sample size, normality, and constant variance, which single‐case data cannot meet (Brossart et al, 2014).…”

Structured visual analysis of single‐case experimental design data: Developments and technological advancements

“…This approach was designed to delineate tools or criteria used for structured visual analysis rather than general quantitative approaches to assess the magnitude of SCED data. As such, effect size indices were not considered as a structured visual analysis tool or criteria in the search (see Dowdy, Peltier et al, 2021 for a review).…”