Using Data to Improve Fidelity When Implementing Evidence-Based Programs

Abstract

In fall 2011, the South Carolina Campaign to Prevent Teen Pregnancy (SC Campaign), with funding from Office of Adolescent Health, began replicating an evidence-based curriculum, It's Your Game, Keep It Real in 12 middle schools across South Carolina. Fidelity of the curriculum was monitored by the use of lesson fidelity logs completed by curriculum facilitators and lesson observation logs submitted by independent classroom observers. These data were monitored weekly to identify possible threats to fidelity. The innovative model Fidelity Through Informed Technical Assistance and Training was developed by SC Campaign to react to possible fidelity threats in real time, through a variety of technical assistance modalities. Fidelity Through Informed Technical Assistance and Training guided the 55 hours of technical assistance delivered by the SC Campaign during the first year of It's Your Game, Keep It Real implementation to 18 facilitators across 12 SC middle schools, and achieved 98.4% curriculum adherence and a high quality of implementation scores.

Evidence-Based Sexuality Education Programs in Schools: Do They Align with the National Sexuality Education Standards?

“…Although each reviewed program demonstrated some behavioral change in participants, it is unclear whether these small behavioral changes are significant enough to offset the costs associated with training educators, program materials, technical assistance, and intervention development and implementation. Finally, there has been some push towards fidelity monitoring of evidence-based sexuality education programs as suggested by the U.S. Department of Health and Human Services (2012) Kershner et al, 2014). However, there remains little empirical evidence that core components of reviewed programs are consistently being implemented faithfully.…”

The impact of multisensory instruction on learning letter names and sounds, word reading, and spelling

“…The two intervention checklists were evaluated for adherence using point-bypoint agreement. The average percent of agreement (Kershener, et al, 2014) indicated structured language interventions were implemented for participants with typical development with 99% (range 94 -100%) fidelity and the multisensory with 98% (range 96 -100%) fidelity. The structured language fidelity for participants with dyslexia was 96% (range 83 -99) and the multisensory fidelity was 96% (87 -99).…”

Testing the Feasibility of Fidelity Evaluation in a Multisite, Multiprogram Initiative

“…Fidelity has been described using five dimensions: adherence, exposure (dose), quality of delivery, participant responsiveness, and program differentiation (Dane & Schneider, 1998;Kershner et al, 2014). Adherence is the degree to which an implementation is delivered as intended by program developers (e.g., whether and to what extent a lesson is modified).…”

“…Although fidelity is often discussed in the context of improving or maintaining positive program outcomes (Carroll et al, 2007;Kershner et al, 2014;Mihalic, 2004), there is also an important professional development, feedback loop, and problem-solving capacity in the monitoring process (Garbers, Flandrick, Bermudez, Meserve, & Chiasson, 2014;Kershner et al;Wiecha, Hannon, & Meyer, 2013). Using fidelity monitoring as part of a process evaluation strategy may assist "organizations reach high levels of implementation fidelity [and] … may also improve program success" (Kershner et al,p.…”

“…Strong fidelity of implementation is associated with improved outcomes of curricula and programs administered in natural settings (such as schools or community centers rather than more tightly controlled experimental settings) and may also be used to enhance professional development for health educators (Benner, Nelson, Stage, & Ralston, 2010;Durlak & DuPre, 2008;Flannery, Fenning, Kato, & McIntosh, 2013;Kershner et al, 2014;Williams & Hunt, 2013).…”