Systematic Protocols for the Visual Analysis of Single-Case Research Data

Wolfe, Katie; Barton, Erin E.; Meadan, Hedda

doi:10.1007/s40617-019-00336-7

Cited by 53 publications

(52 citation statements)

References 27 publications

(46 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Training could terminate when raters show acceptable levels of errors based on untrained exemplars (as in multiple exemplar training). An alternative is to use systematic protocols or procedures (e.g., Manolov & Vannest, 2020;Wolfe et al, 2019) and examine whether they improve the validity of the analyses of raters who use them. Given that behavior analysts put much emphasis on interobserver agreement, calibrating visual inspection appears like the next logical step to improve validity.…”

Section: Discussionmentioning

confidence: 99%

“…The central goal of single-case methodology is to determine whether a causal (i.e., functional) relation exists between the introduction of this independent variable and a change in the dependent variable (Horner & Spaulding, 2010;Cooper et al, 2020). Single-case research focuses on the individual as the relevant unit of analysis and is appropriate for examining intervention effectiveness (Wolfe et al, 2019). Benefits of single-case designs over group designs include the study of low-prevalence disorders (or behaviors) that may be difficult to study with traditional group design procedures that require a large sample size to yield adequate statistical power (Odom et al, 2005).…”

Section: Machine Learning To Analyze Single-case Graphs: a Comparison To Visual Inspectionmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning to Analyze Single-Case Graphs: A Comparison to Visual Inspection

Lanovaz¹,

Hranchuk²

2021

Preprint

View full text Add to dashboard Cite

Behavior analysts commonly use visual inspection to analyze single-case graphs, but studies on its reliability have produced mixed results. To examine this issue, we compared the Type I error rate and power of visual inspection with a novel approach, machine learning. Five expert visual raters analyzed 1,024 simulated AB graphs, which differed on number of points per phase, autocorrelation, trend, variability, and effect size. The ratings were compared to those obtained by the conservative dual-criteria method and two models derived from machine learning. On average, visual raters agreed with each other on only 73% of graphs. In contrast, both models derived from machine learning showed the best balance between Type I error rate and power while producing more consistent results across different graph characteristics. The results suggest that machine learning may support researchers and practitioners in making less error when analyzing single-case graphs, but further replications remain necessary.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Machine Learning To Analyze Single-case Graphs: a Comparison To Visual Inspectionmentioning

confidence: 99%

Machine Learning to Analyze Single-Case Graphs: A Comparison to Visual Inspection

Lanovaz¹,

Hranchuk²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Future research should utilize the single‐case reporting guidelines for intervention research (Horner et al, 2005; Tate et al, 2016). These guidelines include the procedures for conducting visual analysis, including consistency of level, trend, and variability within each phase, immediacy of effect, and overlap with prior phases (Wolfe et al, 2019). This will be important to address in any follow‐up studies involving behavioral intervention methodology, such as PRT or ABA, using a multiple baseline or A–B–A–B design for single‐case‐research.…”

Section: Discussionmentioning

confidence: 99%

Pivotal response treatment for autism: A brief report on training for rural communities

Dahiya

Openden

Ostmeyer

et al. 2021

Journal Community Psychology

View full text Add to dashboard Cite

Many providers from rural communities feel ill‐prepared to treat children with autism spectrum disorder (ASD). Cost‐effective training in Pivotal Response Treatment (PRT), an evidence‐based ASD treatment, can address unmet needs for rural communities. The current study examined a 1‐day general PRT workshop for parents and professionals followed by a 3‐day small, intensive training for professionals. Fifty‐two parents and providers completed surveys before and after Day 1, indicating improvements in perceived stress and confidence. Three providers were trained during Days 2–4, and submitted four 10‐min videos (i.e., baseline, 1‐week, 1‐month, and 2‐months post‐training) working with a target child discussed in the training and another child. Videos were coded for correct PRT implementation and analyzed using a single‐subject A–B design with generalization and maintenance probes. All providers learned to apply PRT techniques with the target child and generalized skills to another child within 1‐week post‐training, with partial maintenance. Implications for rural mental health training in evidence‐based practices are discussed.

show abstract

“…It may not be straightforward to define what an effect is when performing a visual analysis (see Wolfe et al, 2019), but we will not discuss this here, given that the focus is on multilevel models. In terms of quantifying, it may be more straightforward to objectively define an "effect", but it is still not a flawless process.…”

Section: Defining What An "Effect" Ismentioning

confidence: 99%

Assessing consistency of effects when applying multilevel models to single-case data

Manolov

Ferron

2020

Behav Res

View full text Add to dashboard Cite

In the context of single-case experimental designs, replication is crucial. On the one hand, the replication of the basic effect within a study is necessary for demonstrating experimental control.On the other hand, replication across studies is required for establishing the generality of the intervention effect. Moreover, the "replicability crisis" presents a more general context further emphasizing the need for assessing consistency in replications. In the current text, we focus on replication of effects within a study and we specifically discuss the consistency of effects. Our proposal for assessing the consistency of effects refers to one of the promising data analytical techniques: multilevel models, also known as hierarchical linear models or mixed effects models.One option is to check, for each case in a multiple-baseline design, whether the confidence interval for the individual treatment effect excludes zero. This is relevant for assessing whether the effect is replicated as being non-null. However, we consider that it is more relevant and informative to assess, for each case, whether the confidence interval for the random effects includes zero (i.e., whether the fixed effect estimate is a plausible value for each individual effect). This is relevant for assessing whether the effect is consistent in size, with the additional requirement that the fixed effect itself is different from zero. The proposal for assessing consistency is illustrated with real data and it is implemented in free user-friendly software.

show abstract

Systematic Protocols for the Visual Analysis of Single-Case Research Data

Cited by 53 publications

References 27 publications

Machine Learning to Analyze Single-Case Graphs: A Comparison to Visual Inspection

Machine Learning to Analyze Single-Case Graphs: A Comparison to Visual Inspection

Pivotal response treatment for autism: A brief report on training for rural communities

Assessing consistency of effects when applying multilevel models to single-case data

Contact Info

Product

Resources

About