The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs

Abstract

Background Most implementations fail before the corresponding services are ever delivered. Measuring implementation process fidelity may reveal when and why these attempts fail. This knowledge is necessary to support the achievement of positive implementation milestones, such as delivering services to clients (program start-up) and competency in treatment delivery. The present study evaluates the extent to which implementation process fidelity at different implementation stages predicts achievement of those milestones. Methods Implementation process fidelity data—as measured by the Stages of Implementation Completion (SIC)—from 1287 implementing sites across 27 evidence-informed programs were examined in mixed effects regression models with sites nested within programs. Implementation process fidelity, as measured by the proportion of implementation activities completed during the three stages of the SIC Pre-Implementation phase and overall Pre-Implementation (Phase 1) and Implementation (Phase 2) proportion scores, was assessed as a predictor of sites achieving program start-up (i.e., delivering services) and competency in program delivery. Results The predicted probability of start-up across all sites was low at 35% (95% CI [33%, 38%]). When considering the evidence-informed program being implemented, that probability was nearly twice as high (64%; 95% CI [42%, 82%]), and 57% of the total variance in program start-up was attributable to the program. Implementation process fidelity was positively and significantly associated with achievement of program start-up and competency. The magnitude of this relationship varied significantly across programs for Pre-Implementation Stage 1 (i.e., Engagement) only. Compared to other stages, completing more Pre-Implementation Stage 3 (Readiness Planning) activities resulted in the most rapid gains in probability of achieving program start-up. The predicted probability of achieving competency was very low unless sites had high scores in both Pre-Implementation and Implementation phases. Conclusions Strong implementation process fidelity—as measured by SIC Pre-Implementation and Implementation phase proportion scores—was associated with sites’ achievement of program start-up and competency in program delivery, with early implementation process fidelity being especially potent. These findings highlight the importance of a rigorous Pre-Implementation process.

Planting Seeds for Food Is Medicine: Pre-Implementation Planning Methods and Formative Evaluation Findings From a Multi-Clinic Initiative in the Midwest

“…The FIM expert team who led training during the pre-implementation phase have transitioned to leading the evaluation of the implementation, which will include evaluation of the essential stages of “start-up” implementation, including the status of staff hiring/training, fidelity monitoring, launch of services, and ongoing program delivery. 24 Since all clinics independently decided to narrow their population of interest to patients with type 2 diabetes or pre-diabetes, the evaluation design will employ shared outcome metrics from similar populations across sites (pre- and post-intervention food security, F/V and fiber intake, A1c and other health outcomes). Implementation evaluation will draw from a combination of qualitative (provider, staff, and patient interviews) as well as quantitative data sources to ensure full assessment of the barriers and facilitators to project effectiveness and sustainability.…”

“…Despite many types of evidence-informed interventions and programs that are available to the healthcare sector, the estimated start-up (initial implementation) rate of such initiatives is quite low. 24 Pre-implementation planning may help to address this challenge and involves naming and framing of the problem, identification of stakeholders, and consideration of evidence-based practices as essential prerequisites prior to the implementation of a new clinic-based initiative. 25 , 26 During this process, pre-implementation occurs through a series of stages including engagement, consideration of feasibility, and readiness planning.…”

“… 25 , 26 During this process, pre-implementation occurs through a series of stages including engagement, consideration of feasibility, and readiness planning. 24 These important steps not only help to ensure essential elements are in place for the immediate viability of new programs (ie, ensuring readiness for organizational change), but they also can help to lay the foundation for the long-term sustainability of programs as well as help an organization decide whether its in their best interest to move ahead with implementation of the program. Participation in pre-implementation planning is strongly associated with the start-up of a new evidence-informed programs, as well as mastering full competency in delivery of that program.…”

“…Participation in pre-implementation planning is strongly associated with the start-up of a new evidence-informed programs, as well as mastering full competency in delivery of that program. 24 Pre-implementation planning is arguably most critical when designing new patient services that require cross-sector collaborations, such as food systems procurement of specific foods to meet individualized nutrition needs of patients with complex health conditions.…”

Implementing a pharmacist-integrated collaborative model of medication treatment for opioid use disorder in primary care: study design and methodological considerations

“…This tool has been adapted for many evidence-based practices including mental health, school prevention programs, primary care interventions, substance use disorder treatments, and large state system initiatives; a “universal” tool has been developed for use of monitoring of general implementation strategies [ 50 ]. Results from studies using the SIC have repeatedly shown that when there is variability among the strategies sites use to implement a new practice, actions taken during the implementation process predict the achievement of program start-up and competency as measured with this tool [ 41 , 51 ]. The SIC was tailored for this protocol using standardized SIC adaptation procedures [ 41 ] by the SIC developers, PrIMO model developers, and members of the research team.…”