Application of automated nonparametric statistical analysis in clinical contexts

Abstract

Functional analyses (FAs) provide clinicians with results upon which they design behavioral treatments. Unfortunately, interrater reliability of visual analysis of FA results can be inconsistent. Accordingly, researchers have designed quantitative metrics and visual aids to supplement visual analysis. Recently, Hall et al. ( 2020) provided a proof of concept for using automated nonparametric statistical analysis (ANSA) to interpret FA data. Their results show promise for ANSA as a supplemental tool. However, they evaluated ANSA with only published FA datasets, which may not be representative of FAs commonly encountered in clinical care. Therefore, the purpose of this replication was to compare ANSA to another validated supplemental aid (i.e., the structured criteria method) and investigate its utility with unpublished clinical FA data. Our results were consistent with Hall et al.'s, indicating ANSA may augment clinical interpretation of FA data. Recommendations for clinical applications of ANSA and future directions for researchers are discussed.

Quantification of behavioral data with effect sizes and statistical significance tests

“…Skinner (1938) and Sidman (1960) emphasized experimental control with replicable methods and results over statistical analysis. However, quantitative approaches have been adopted and used in behavior analysis and behavior science at large (Becraft et al, 2020; Falligant et al, 2020; Gilroy et al, 2017; Kranak et al, 2021; Kyonka, 2019; Young, 2018; 2019) while experimental control and replication remains the most important area of emphasis (Hagopian, 2020; Kazdin, 2020). The addition of significance tests and effect sizes provides benefits for clinical decision‐making, communication across sciences, and allows for inclusion in systematic reviews, an area of growing importance and attention in single‐case research (see Dowdy et al, 2021; Manolov et al, 2017; Parker & Hagan‐Burke, 2007; Shadish et al, 2016).…”

A proposal for the assessment of replication of effects in single‐case experimental designs

“…The researcher makes choices by clicking or by writing a few numbers but nothing else is required. The software is an application built via Shiny, as for other data analytical approaches (e.g., Declercq et al, 2020; Kranak et al, 2021), which makes training of users to construct the graphs for themselves following multiple steps unnecessary (e.g., Dixon et al, 2009; Lehardy et al, 2021; Mitteer et al, 2018).…”

“…For instance, the conservative dual criterion (Fisher et al, 2003) involves representing mean and trend, and the application of statistical process control (Callahan & Barisa, 2005;Pfadt & Wheeler, 1995) involves representing mean and variability. Similarly, a proposal for making the functional analysis of behavior automatic also entails multiple data features (Kranak et al, 2021). Additionally, in a forest plot as a graphical representation commonly used in 1017 Assessing Replication of Effects meta-analyses, the size of the dot or square representing an effect reflects its precision and is related to the number of measurements that it is based on (Anzures-Cabrera & Higgins, 2010).…”

A Proposal for the Assessment of Replication of Effects in Single-Case Experimental Designs

“…The researcher makes choices by clicking or by writing a few numbers but nothing else is required. The software is an application built via Shiny, as for other data analytical approaches (e.g., Declercq et al, 2020;Kranak et al, 2021), which makes training of users to construct the graphs for themselves following multiple steps unnecessary (e.g., Dixon et al, 2009;Lehardy et al, 2021;Mitteer et al, 2018).…”

“…For instance, the conservative dual criterion (Fisher et al, 2003) involves representing mean and trend, and the application of statistical process control (Callahan & Barisa, 2005;Pfadt & Wheeler, 1995) involves representing mean and variability. Similarly, a proposal for making the functional analysis of behavior automatic also entails multiple data features (Kranak et al, 2021). Additionally, in a forest plot as a graphical representation commonly used in 1017 Assessing Replication of Effects 19383703, 2022, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jaba.923 by Readcube (Labtiva Inc.), Wiley Online Library on [03/01/2023].…”