Monte Carlo Simulations to Develop Guidelines for the Use of Single-Case Designs: A Tutorial

Abstract

Despite being a cornerstone of the science of behavior analysis, researchers and practitioners often rely on tradition and consensus-based guidelines, rather than empirical evidence, to make decisions about single-case designs. One approach to develop empirically-based guidelines is to use Monte Carlo simulations for validation, but behavior analysts are not necessarily trained to apply this type of methodology. Therefore, the purpose of our technical article is to walk the reader through conducting Monte Carlo simulations to examine the accuracy, Type I error rate, and power of a visual aid for AB graphs using R Code. Additionally, the tutorial provides code to replicate the procedures with single-case experimental designs as well as with the Python programming language. Overall, a broader adoption of Monte Carlo simulations to validate guidelines should lead to an improvement in how researchers and practitioners use single-case designs.

The influence of data characteristics on interrater agreement among visual analysts

“…These participants may not have been representative of the population of visual analysts at large; however, given the number of graphs needed to include multiple examples of all combinations of characteristics, it was likely prohibitively costly to have similar numbers of participants as in other visual analysis studies that have used a much smaller set of graphs (e.g., 31 graphs rated by 52 experts; Wolfe et al, 2016). We agree with Lanovaz (2022) that there is value in having fewer raters analyze a larger set of graphs to allow for the controlled investigation of the effects of data characteristics. Nonetheless, similar procedures should be replicated with additional raters and additional graphs in future research to determine whether the effects reported here remain consistent.…”