Fidelity to the ACT SMART Toolkit: An Instrumental Case Study of Implementation Strategy Fidelity

Abstract

Background: Although evidence-based practices (EBPs) have been shown to improve a variety of outcomes for autistic children, they are often inconsistently implemented or not implemented in community settings where many autistic children receive usual care services. One blended multi-phased implementation strategy developed to support the adoption and implementation of EBPs for autism spectrum disorder (ASD) in community-based settings is the Autism Community Toolkit: Systems to Measure and Adopt Research-Based Treatments (ACT SMART Toolkit). Guided by an adapted version of the EPIS, the ACT SMART Toolkit is comprised multiple phases, each with steps and activities to be completed by agency implementation teams. In the present study, we evaluated implementation strategy fidelity, given the potential for important insights into the use of the toolkit and limited information on the phenomenon of implementation strategy fidelity more broadly.Methods: We used an instrumental case study approach to assess fidelity to the ACT SMART Toolkit during its pilot study with six ASD community agency implementation teams. We assessed adherence, dose, and implementation team responsiveness for each phase and activity of the toolkit at both an aggregate and individual agency level. We also conducted repeated measures ANOVAs to determine whether implementation strategy fidelity significantly differed by toolkit phase. Results: Overall, we found that adherence, dose, and participant responsiveness to the ACT SMART Toolkit were high, with some variability by toolkit phase and activity, as well as by ASD community agency. There was a significant main effect of toolkit phase for dose (F(2,8) = 10.93, MSE = .190, p = .005, η2 = .73), such that dose was significantly lower during the preparation phase as compared to the implementation phase of the toolkit. Conclusions: Our instrumental case study evaluation of fidelity to the ACT SMART Toolkit demonstrated potential for the strategy to be used with fidelity in ASD community-based agencies. Findings related to variability of implementation strategy fidelity in the present study may also inform future adaptations to the toolkit and point to broader trends of how implementation strategy fidelity may vary by content and context.