Ensuring Implementation Fidelity of a 10-Week Home Visiting Program in Two Randomized Clinical Trials

Abstract

Introduction Implementation fidelity is a challenge for the adoption of evidence-based programs within social service broadly and child welfare specifically. However, implementation fidelity is critical for maintaining the integrity of clinical trials and for ensuring successful delivery of services in public health settings. Methods Promoting First Relationships (PFR), a 10-week home visiting parenting intervention, was evaluated in two randomized clinical trials with populations of families in child welfare. Seven providers from community agencies participated in the trials and administered PFR. Fidelity data collected included observational measures of provider behavior, provider records, and input from clients to assess training uptake, adherence to content, quality of delivery, program dosage, and participant satisfaction. Results In mock cases to assess training uptake, providers demonstrated an increase in PFR verbalization strategies and a decrease non-PFR verbalizations from pre to post PFR training, and overall this was maintained a year later (Mann-Whitney U's = 0, p's< .01). Adherence to content in actual cases was high, with M = 97% of the program elements completed. Quality of delivery varied across providers, indicated by PFR consultation strategies (Wilks' Lambda F = 18.24, df = 15, p < .001) and global ratings (F = 13.35, df = 5, p < .001). Program dosage was high in both trials (71 and 86% receiving 10 sessions), and participant satisfaction was high (M = 3.9, SD = 0.2; 4 = greater satisfaction). Discussion This system of training and monitoring provides an example of procedures that can be used effectively to achieve implementation fidelity with evidence-based programs in social service practice.

Does Parents’ Own History of Child Abuse Moderate the Effectiveness of the Promoting First Relationships® Intervention in Child Welfare?

“…Parent–child dyads ( n = 124) randomized to the PFR intervention were offered 10 weekly 60- to 75-min in-home visits by one of the two PFR providers from a community agency who had a master’s degree in social work or counseling. Training of providers and procedures to ensure implementation fidelity are described in detail in Oxford et al (2018). Attachment theory–informed and strength-based consultation strategies focused on increasing parental sensitivity to child cues.…”

Promoting First Relationships®

“…The providers completed PFR training over a 5-month period and, after becoming certified, began meeting with study families. Throughout the study, fidelity was monitored by a PFR master trainer (Oxford, Spieker, Lohr, Fleming, Dillon, & Rees, 2016). The providers submitted videotapes of themselves working with the families (one videotaped session for each family on their caseload), which the master trainer used to rate the quality of delivery on a 1- to 5-point scale.…”

“…The response rate was 78.5%, and the internal consistency of the scale was good (α = .77). PFR participants rated PFR very highly ( M = 3.9, SD = 0.2; range = 1–4, higher scores = greater satisfaction), and all would recommend the program (Oxford, Spieker, et al, 2016).…”

What works for whom? Mother's psychological distress as a moderator of the effectiveness of a home visiting intervention

“…PFR is a strengths‐based program using video feedback. Providers are trained to video record parent‐child interaction for 10–15 min and, on alternating weeks, reflect on the unedited recording with the parent and use PFR consultation strategies to support parental reflection, strengths, and confidence (for details see Oxford et al., 2018). Reflection is supported when the provider asks a reflective question based on the video observation, such as, “What do you think your child was feeling when she arched her back?” The parent responds with their thoughts, which opens a dialogue between provider and parent.…”