Quantifying false positives in simulated events using partial interval recording and momentary time sampling with dual‐criteria methods

Abstract

The dual-criteria (DC) and conservative dual-criteria (CDC) methods allow clinicians and researchers to quantify the occurrence of false-positive outcomes within single-case experimental designs. The purpose of the current study was to use these DC and CDC methods to measure the incidence of false positives with simulated data collected via discontinuous interval methods (i.e., momentary time sampling, partialinterval recording) as a function of data series length. We generated event data to create 316,800 unique simulated data series for analysis. In Experiment 1, we evaluated how changes in relevant parameters (i.e., interval sizes, event durations, IRT-to-event-run ratios) produced false positives with momentary time sampling procedures. We also assessed the degree that the CDC method produced fewer false positives than the DC method with simulated interval data. In Experiment 2, we used similar procedures to quantify the occurrence of false positives with partial-interval recording data. We successfully replicated outcomes from previous research in the current study, though such results only highlight the generality of the procedures relating to false positive (and not false negative) outcomes. That is, these results indicate MTS and PIR may adequately control for false positives, but our conclusions are limited by a lack of data on power. K E Y W O R D S dual-criteria method, false positives, momentary time sampling, partial interval recording

Single-case design effect-size distributions: Association with procedural parameters.

“…Specifically, data from both hypothetical (e.g., simulated) and existent single-case studies suggest that partial interval recording (PIR) may result in reasonable estimates of count, but only under certain conditions (Pustejovsky & Swan, 2015; Yoder et al, 2018), while momentary time sampling (MTS) generally results in better estimates of duration (Ledford et al, 2015; Radley et al, 2015). However, evaluations of differential outcomes based on measurement systems have focused on functional relation determination and absolute rather than relative behavior changes between conditions (e.g., Falligant & Vetter, 2020; Meany-Daboul et al, 2007; Schmidt et al, 2013). For example, Ledford et al (2015) established that PIR with 5-s intervals might result in overestimates of behavior change because the absolute overestimation at a lower frequency (e.g., an estimate of 20 when the behavior occurred 10 times per session) was lower than the overestimation for a higher frequency (e.g., an estimate of 113 when the behavior actually occurred 70 times per session).…”