Fidelity of implementation: development and testing of a measure

Abstract

BackgroundAlong with the increasing prevalence of chronic illness has been an increase in interventions, such as nurse case management programs, to improve outcomes for patients with chronic illness. Evidence supports the effectiveness of such interventions in reducing patient morbidity, mortality, and resource utilization, but other studies have produced equivocal results. Often, little is known about how implementation of an intervention actually occurs in clinical practice. While studies often assume that interventions are used in clinical practice exactly as originally designed, this may not be the case. Thus, fidelity of an intervention's implementation reflects how an intervention is, or is not, used in clinical practice and is an important factor in understanding intervention effectiveness and in replicating the intervention in dissemination efforts. The purpose of this paper is to contribute to the understanding of implementation science by (a) proposing a methodology for measuring fidelity of implementation (FOI) and (b) testing the measure by examining the association between FOI and intervention effectiveness.MethodsWe define and measure FOI based on organizational members' level of commitment to using the distinct components that make up an intervention as they were designed. Semistructured interviews were conducted among 18 organizational members in four medical centers, and the interviews were analyzed qualitatively to assess three dimensions of commitment to use--satisfaction, consistency, and quality--and to develop an overall rating of FOI. Mixed methods were used to explore the association between FOI and intervention effectiveness (inpatient resource utilization and mortality).ResultsPredictive validity of the FOI measure was supported based on the statistical significance of FOI as a predictor of intervention effectiveness. The strongest relationship between FOI and intervention effectiveness was found when an alternative measure of FOI was utilized based on individual intervention components that had the greatest variation across medical centers.ConclusionsIn addition to contextual factors, implementation research needs to consider FOI as an important factor in influencing intervention effectiveness. Our proposed methodology offers a systematic means for understanding organizational members' use of distinct intervention components, assessing the reasons for variation in use across components and organizations, and evaluating the impact of FOI on intervention effectiveness.

The “Critical” Elements of Illness Management and Recovery: Comparing Methodological Approaches

“…However, these elements are often not self-evident. Identification of critical elements is a precursor for measuring program fidelity (also known as treatment integrity; Carroll et al, 2007; Keith, Hopp, Subramanian, Wiitala, & Lowery, 2010; Kitson, Harvey, & McCormack, 1998; Mihalic, 2004; Perepletchikova, Treat, & Kazdin, 2007). Treatment fidelity is a key indicator of construct validity in research (Cook, Campbell, & Day, 1979), and higher fidelity has been associated with better outcomes across a variety of interventions (e.g., Baer et al, 2007; Durlak & DuPre, 2008; Dusenbury, Brannigan, Falco, & Hansen, 2003; Haddock et al, 2001; McGrew & Griss, 2005; however, see Bond & Salyers (2004) and Webb, Derubeis, and Barber (2010) for counterexamples).…”

The CareWell in Hospital program to improve the quality of care for frail elderly inpatients: results of a before–after study with focus on surgical patients

“…We call for ongoing monitoring of processes and outcomes, which is needed for transition experiments such as CWH, and the use of uniform measures of implementation degree to facilitate the continuous monitoring and interpretation of its effects. 36 In addition, regarding outcome measures, Payne et al 37 suggest to lookdin addition to common health-related measuresdat other patient benefits and to incorporate non-health-related outcome measures also, such as meeting expectations, satisfaction with services, perceived personal control, decision making, and so on. For perioperative outcomes research in elderly patients, Peden and Grocott 38 also suggest the use of patient-reported outcome measures or patient-reported experience measures next to traditional outcome measures.…”

Evaluation Protocol To Assess an Integrated Framework for the Implementation of the Childhood Obesity Research Demonstration Project at the California (CA-CORD) and Massachusetts (MA-CORD) Sites

“…16 In the dissemination and implementation literature, conceptualizations of implementation effectiveness vary based on the nature of the intervention or organizational change being introduced. [72][73][74][75] In the CORD project, dimensions of implementation effectiveness identified as critical mediators of intervention outcomes include exposure (dose and reach), quality of delivery, fidelity, participant responsiveness, and differentiation. A detailed definition of each of these dimensions is provided in Table 3.…”