Examining the equivalence of fidelity over two generations of KEEP implementation: A preliminary analysis

Abstract

Problem In order to obtain and maintain positive outcomes garnered from evidence-based practice (EBP) models, it is necessary to implement them effectively in “real world” settings, to continually monitor intervention fidelity to prevent drift, and to train new staff due to turnover. The fidelity monitoring processes that are commonly employed in research settings are labor intensive and probably unrealistic to employ in community agencies given the additional burden and cost that they represent over and above the cost of implementing the EBP. Efficient strategies for implementing fidelity monitoring and staff training procedures within the inner context of agency settings are needed to promote agency self-sufficiency and program sustainability. Method A cascading implementation model was used whereby agencies who achieved proficiency in KEEP, an EBP designed to prevent placement disruptions in foster and kinship child welfare homes, were trained to take on fidelity management roles to improve the likelihood of program sustainability. Agency staff were trained to self-monitor fidelity and to train internal staff to achieve model fidelity. A web-based system for conducting fidelity assessments and for onsite/internal and remote program quality monitoring was utilized. Results Scores on fidelity ratings from streamed observations of intervention sessions showed no differences for foster parents treated by first generation interventionists trained by model developers compared to a second generation of interventionists trained by the first generation. Conclusion and relevance to child welfare Development of the local intra-agency capacity to manage quality intervention delivery is an important feature of successful EBP implementation. Use of the cascading implementation model appears to support the development of methods for effective monitoring of fidelity of the KEEP intervention, for training new staff, and ultimately for the development of internal methods for maintaining program sustainability and effectiveness.

Implementation and evaluation of linked parenting models in a large urban child welfare system

“…Thus, OSLC sought to build a mechanism whereby the agencies were prepared to use their own internal personnel to take over the implementation activities that initially OSLC developers conducted including the initial staff training workshop and providing ongoing fidelity monitoring and the consulting activities. Drawing on our previous experiences in creating sustainability plans for the KEEP model (Chamberlain, Price, Reid, et al, 2008), we opted to use the Cascading Dissemination model (Buchanan, Chamberlain, Price, & Sprengelmeyer, 2013). …”

“…Fidelity is measured across three dimensions including (1) content, (2) process, and (3) structure using a standardized rating protocol (Facilitator Adherence Rating; FAR, Buchanan et al, 2013). During groups, the facilitators record each session using a laptop with software that enables the recording to be uploaded to a secure website.…”

Odd Couple? Reenvisioning the Relation Between Science and Practice in the Dissemination-Implementation Era

“…As one possible remedy, some researchers have explored the impact of various 'train the trainer" models in which indigenous agency staff are taught to provide the EBP training and/or supervision after the outside experts are gone. In their cascading implementation research described previously, Chamberlain and colleagues found that "second generation" interventionists (trained and supervised by "first generation" interventionists who had been trained by the treatment developers) achieved fidelity and client outcomes similar to those obtained by the first generation interventionists themselves (Buchanan, Chamberlain, Price & Sprengelmeyer, 2013;Chamberlain, Price, Reid, & Landsverk, 2008). Other treatment developers have had less positive experiences with train-the-trainer and second generation models, and have thus retained more prolonged involvement of the intervention developer team (see e.g., Curtis, Ronan, & Borduin, 2004;Henggeler & Schaeffer, 2010).…”

Toward Creating Synergy Among Policy, Procedures, and Implementation of Evidence-Based Models in Child Welfare Systems: Two Case Examples

“…Fidelity to the KEEP curriculum is monitored closely and is measured across three dimensions including: (1) content, (2) process, and (3) structure, using a standardized rating protocol (Facilitator Adherence Rating; FAR; Buchanan, Chamberlain, Price, & Sprengelmeyer, 2013). During groups, the facilitators record each session using a laptop with software that enables the recording to be uploaded to a HIPAA-compliant, secure website designed to allow for direct observation of group sessions (Fidelity Observation System; FIDO).…”