Masked Visual Analysis: Minimizing Type I Error in Visually Guided Single-Case Design for Communication Disorders

Abstract

Purpose Single-case experimental designs are widely used to study interventions for communication disorders. Traditionally, single-case experiments follow a response-guided approach, where design decisions during the study are based on participants' observed patterns of behavior. However, this approach has been criticized for its high rate of Type I error. In masked visual analysis (MVA), response-guided decisions are made by a researcher who is blinded to participants' identities and treatment assignments. MVA also makes it possible to conduct a hypothesis test assessing the significance of treatment effects. Method This tutorial describes the principles of MVA, including both how experiments can be set up and how results can be used for hypothesis testing. We then report a case study showing how MVA was deployed in a multiple-baseline across-subjects study investigating treatment for residual errors affecting rhotics. Strengths and weaknesses of MVA are discussed. Conclusions Given their important role in the evidence base that informs clinical decision making, it is critical for single-case experimental studies to be conducted in a way that allows researchers to draw valid inferences. As a method that can increase the rigor of single-case studies while preserving the benefits of a response-guided approach, MVA warrants expanded attention from researchers in communication disorders.

The Effects of a Multicomponent Self-Monitoring Intervention on the Rates of Pre-Service Teacher Behavior-Specific Praise in a Masked Single-Case Experimental Design

“…The first and second authors examined the visual representation of the data to determine the presence or absence of a functional relation by focusing on mean-level change, trend, and variability within and between phases, and immediacy of change at the intercept gap (Kratochwill et al, 2010; Ledford & Gast, 2018). Per current practices and recommendations, a masked visual analysis was then completed by three independent raters with no knowledge of the study (Byun et al, 2017; Vannest et al, 2018). Raters evaluated the data visually after a brief training of definitions (i.e., trend, overlap) and scored the data using a coding sheet.…”

SCD‐MVA : A mobile application for conducting single‐case experimental design research during the pandemic

“…Figure 1 gives a graphical display of an MBD across four participants (i.e., Samantha, Frank, Timothy, and Alejandro). The graphical display in Figure 1 was created using a subset of the data gathered by Byun, Hitchcock, and Ferron (2017). Byun et al investigated the influence of an intervention (i.e., visual‐acoustic biofeedback) on a dependent variable (i.e., percent of correct syllable‐level tokens).…”

“…The data that will be used for the empirical demonstration of the MVA approach and the mobile application is retrieved from a published study in the domain of communication disorders. The study was published in 2017 in the Journal of Speech, language, and Hearing Research (Byun et al, 2017). Byun et al graphically displayed the gathered data.…”

“…Byun et al (2017) investigated an intervention for residual errors effecting/r/. More specifically, the intervention is visual‐acoustic biofeedback and the dependent variable is percent of syllable‐level tokens rated correct by blinded listeners.…”

“…After baseline stability criteria are met, the intervention can start for one of the participants. The graphical display of the data from the study of Byun et al (2017) is presented in Figure 3. In that graph, you can see that the first participant, Alexandro was the last participant to be exposed to the intervention (based on the randomization schedule that was determined prior to the start of the experiment).…”