Examining How Treatment Fidelity Is Supported, Measured, and Reported in K–3 Reading Intervention Research

Abstract

Treatment fidelity data (descriptive and statistical) are critical to interpreting and generalizing outcomes of intervention research. Despite recommendations for treatment fidelity reporting from funding agencies and researchers, past syntheses have found treatment fidelity is frequently unreported (e.g., Swanson, The Journal of Special Education, 47, 3–13, 2011) in educational interventions and fidelity data are seldom used to analyze its relation to student outcomes (O’Donnell, Review of Educational Research, 78(1), 33–84, 2008). The purpose of this synthesis was to examine how treatment fidelity is supported, measured, and reported in reading intervention studies conducted with students at risk or with reading difficulties in grades K–3 from 1995 through 2015. All studies (k = 175) were coded to extract and classify information related to (a) the characteristics of the intervention study (e.g., publication year, research design); (b) treatment implementer training and support; (c) treatment fidelity data collection procedures, dimensions (i.e., adherence, quality, receipt, dosage, and differentiation), and levels of treatment fidelity data; and (d) the use of fidelity scores in the analysis of treatment effects. Results indicated that less than half (47%) of the reading intervention studies synthesized reported treatment fidelity data (numeric or narrative). Exploratory analyses showed that several study features were associated with the prevalence of fidelity reporting. Studies reporting treatment fidelity largely measured treatment adherence, and scores were, on average, high. Other dimensions of treatment fidelity (e.g., treatment differentiation), and analyses relating fidelity data to outcomes, were consistently absent from the corpus of reading intervention studies reviewed. Recommendations for enhancing how treatment fidelity data in intervention studies are collected and reported are presented.

Illuminating learning from informative texts in secondary education: A switching replication design study

“…In line with previous research (Bouwer, Koster, & Van den Bergh, 2017;Author et al, et al, 2018d;De Smedt, Graham, & Van Keer, 2019), three methods were applied to ensure fidelity of implementation (Capin, Walker, Vaughn, & Wanzek, 2017;O'Donnell. , 2008): (1) students' workbooks were verified, (2) teachers completed logbooks during the intervention, and (3) three researchers conducted observations in all classes during both phases.…”

Art Therapy for Children Diagnosed with Autism Spectrum Disorders: Development and First Evaluation of a Treatment Programme

“…It is not clear if it is to be expected that expanding the duration of 'Images of Self' would improve the outcomes. Empirical evidence of the treatment programme can be enhanced by applying a treatment fidelity study (Capin, Walker, Vaughn, & Wanzek, 2018). This involves a more detailed research of the competencies and behaviour of the art therapist, of the contribution of parents and teachers, and the contribution of art materials and the art expressions of the children.…”

Tracking Intervention Dosage to Inform Instructional Decision Making

“…Quality indicators in special education research call for intervention dosage data to be collected and reported as part of the research process (Cook et al, 2015), yet recent research reveals intervention dosage is scarcely reported (Capin et al, 2018). In their systematic review, Capin et al (2018) examined the current landscape of reading intervention literature to uncover how, among other things, authors attended to issues of implementation fidelity across a 10-year time frame. Of the 83 studies that reported implementation fidelity, 14 studies reported using multiple implementation fidelity measures, and of those 14 studies, only two included measures of intervention dosage.…”

“…In light of the scarcity of collected or reported intervention dosage data, the authors called for the inclusion of multiple dimensions of implementation fidelity so that teachers could gain a clearer picture of how the intervention was implemented and what effects it had on student outcomes. The Capin et al (2018) study points to the importance of intervention dosage, especially as it relates to the work of classroom teachers.…”

“…There are two primary challenges to collecting and analyzing intervention dosage data. First, within existing intervention research, when intervention dosage data are collected, they are reported at the teacher level (i.e., time the teacher delivered the intervention) rather than at the student level (i.e., time individual students actually received the intervention; Capin et al, 2018).…”