Measuring Implementation Fidelity in a Community-Based Parenting Intervention

Abstract

Background Establishing the feasibility and validity of implementation fidelity monitoring strategies is an important methodological step in implementing evidence-based interventions on a large scale. Objectives The objective of the study was to examine the reliability and validity of the Fidelity Checklist, a measure designed to assess group leader adherence and competence delivering a parent training intervention (the Chicago Parent Program) in child care centers serving low-income families. Method The sample included 9 parent groups (12 group sessions each), 12 group leaders, and 103 parents. Independent raters reviewed 106 audiotaped parent group sessions and coded group leaders’ fidelity on the Adherence and Competence Scales of the Fidelity Checklist. Group leaders completed self-report adherence checklists and a measure of parent engagement in the intervention. Parents completed measures of consumer satisfaction and child behavior. Results High interrater agreement (Adherence Scale = 94%, Competence Scale = 85%) and adequate intraclass correlation coefficients (Adherence Scale = .69, Competence Scale = .91) were achieved for the Fidelity Checklist. Group leader adherence changed over time, but competence remained stable. Agreement between group leader self-report and independent ratings on the Adherence Scale was 85%; disagreements were more frequently due to positive bias in group leader self-report. Positive correlations were found between group leader adherence and parent attendance and engagement in the intervention and between group leader competence and parent satisfaction. Although child behavior problems improved, improvements were not related to fidelity. Discussion The results suggest that the Fidelity Checklist is a feasible, reliable, and valid measure of group leader implementation fidelity in a group-based parenting intervention. Future research will be focused on testing the Fidelity Checklist with diverse and larger samples and generalizing to other group-based interventions using a similar intervention model.

Development and validation of the ASPIRE-VA coaching fidelity checklist (ACFC): a tool to help ensure delivery of high-quality weight management interventions

“…Though a few recent studies have developed fidelity assessment tools based on a self-reported measure of behavioral health provider fidelity [45] or expert ratings for delivering a complex intervention for chronic pain [46] and a practice change intervention [47] (both of the latter instruments assessed face validity and reliability only), the dearth of validated and published instruments extends beyond weight loss interventions and is well-recognized [48,49,9,[50][51][52][53]. Clearly, there is a need for validated measures to help determine the effectiveness of weight loss interventions when delivered as they were intended.…”

Factors associated with the implementation of the Familias Unidas intervention in a type 3 translational trial

“…For Hispanic families, unique risk factors associated with immigration (e.g., adapting to a new culture, language barriers) may compromise family functioning, an important buffer against negative adolescent behavioral outcomes [20]. Characteristics of the family may also impact intervention attendance, which has been positively associated with ratings of fidelity [21]. In previous Familias Unidas efficacy trials, family characteristics (e.g., stress, income, and adolescent behavior problems) were significantly associated with initial engagement to the intervention [22].…”

A Prenatal Coparenting Intervention With Unmarried Father–Mother Dyads: Fidelity of Intervention Delivery by Male–Female Community Mentor Teams

“…Adjustments, informed by the way the Fidelity Checklist items overlapped with FIOC curricular content and intent, were made by the first author working with a doctoral student who later served as one of the QAAs (see Table 1 for a summary of the adjusted items and mean competence scores). As an example of item adjustment, Breitenstein et al (2010) worked with a group of parents and created the study instrument for these sessions. In contrast, in the current study, only one family participated in the sessions, so the original Fidelity Checklist item ''actively engages all parents in the discussion'' was reworded as ''actively engages both parents in the discussions and activities.''…”

“…ICCs were .79 and .55 for the adherence and competence scores, respectively. Internal consistency for the competence scale was high (Cronbach's alpha = .96), and the high intercorrelation between the competence scale item sets adapted from the Breitenstein et al Content/process ratings of ''figuring it out for the child'' sessions To evaluate the emotional tenor of each session, QAAs rated parents' level of conflict (from 0 = no conflict to 4 = very high conflict) and overall tone of the session (quality and affective tone of conversations) on a 9-point scale (1Y3 = negative, 4Y6 = neutral, 7Y9 = positive) on the Fidelity Checklist (Breitenstein et al, 2010).…”

“…Fidelity Checklist, developed by Breitenstein et al (2010): This instrument was originally used to monitor fidelity in the Chicago Parent Program, a communitybased parenting intervention program. For the current study, the 16-item competence scale was used.…”