Implementation and sustainment of diverse practices in a large integrated health system: a mixed methods study

Abstract

Background One goal of health systems seeking to evolve into learning health systems is to accelerate the implementation and sustainment of evidence-based practices (EBPs). As part of this evolution, the Veterans Health Administration (VHA) developed the Innovation Ecosystem, which includes the Diffusion of Excellence (DoE), a program that identifies and diffuses Gold Status Practices (GSPs) across facilities. The DoE hosts an annual “Shark Tank” competition in which leaders bid on the opportunity to implement a GSP with 6 months of implementation support. Over 750 diverse practices were submitted in cohorts 2 and 3 of Shark Tank; 23 were designated GSPs and were implemented in 31 VA networks or facilities. As part of a national evaluation of the DoE, we identified factors contributing to GSP implementation and sustainment. Methods Our sequential mixed methods evaluation of cohorts 2 and 3 of Shark Tank included semi-structured interviews with at least one representative from 30/31 implementing teams ( N = 78/105 people invited) and survey responses from 29/31 teams ( N = 39/47 invited). Interviews focused on factors influencing implementation and future sustainment. Surveys focused on sustainment 1.5–2 years after implementation. The Consolidated Framework for Implementation Research (CFIR) informed data collection and directed content analysis. Ordinal scales were developed inductively to rank implementation and sustainment outcomes. Results Over 50% of teams (17/30) successfully implemented their GSP within the 6-month implementation period. Despite extensive implementation support, significant barriers related to centralized decision-making, staffing, and resources led to partial ( n = 6) or no ( n = 7) implementation for the remaining teams. While 12/17 initially successful implementation teams reported sustained use of their GSP, over half of the initially unsuccessful teams ( n = 7/13) also reported sustained GSP use 1.5 years after the initial implementation period. When asked at 6 months, 18/27 teams with complete data accurately anticipated their future sustainability based on reported sustainment an average of 1.5 years later. Conclusions Most teams implemented within 6 months and/or sustained their GSP 1.5 years later. High levels of implementation and sustainment across diverse practices and teams suggest that VHA’s DoE is a successful large-scale model of diffusion. Team predictions about sustainability after the first 6 months of implementation provide a promising early assessment and point of intervention to increase sustainability.

Rapid versus traditional qualitative analysis using the Consolidated Framework for Implementation Research (CFIR)

“…These EBIs include innovations supported by evidence from research studies and administrative or clinical experience [ 29 , 30 ] and strive to address patient, staff, and/or facility needs. The DoE hosts an annual “Shark Tank” competition, in which VHA leaders compete to implement an EBI with 6 months of external implementation support; for additional detail, see previous publications [ 31 – 34 ]. As part of a national evaluation of the DoE, we identified barriers and facilitators to the implementation of these EBIs in VHA facilities using semi-structured interviews [ 31 ].…”

Rapid Versus Traditional Qualitative Analysis Using the Consolidated Framework for Implementation Research (CFIR)

“…These EBIs include innovations supported by evidence from research studies and administrative or clinical experience [14,15], and strive to address patient, staff, and/or facility needs. The DoE hosts an annual "Shark Tank" competition, in which VHA leaders compete to implement an EBI with 6-months of external implementation support; for additional detail see previous publications [16][17][18][19]. As part of a national evaluation of the DoE, we identi ed barrier and facilitators to implementation of these EBIs in VHA facilities using semi-structured interviews [16].…”

“…The DoE hosts an annual "Shark Tank" competition, in which VHA leaders compete to implement an EBI with 6-months of external implementation support; for additional detail see previous publications [16][17][18][19]. As part of a national evaluation of the DoE, we identi ed barrier and facilitators to implementation of these EBIs in VHA facilities using semi-structured interviews [16]. The qualitative interview and analysis team included CR (MPH, a senior qualitative analyst and CFIR expert user) and AN (PhD, a senior qualitative methodologist and CFIR intermediate user).…”

“…Data collection methods were the same across both approaches; in effect, they will not be discussed in detail in this paper. In brief: we conducted semi-structured telephone interviews with participants involved with implementing an EBI; for additional detail see previous publications [20,21]. Interview guides were informed by the CFIR (see Additional File 1).…”

“…The traditional and rapid qualitative analysis steps used by the team are described in Table 1. The traditional approach is described in detail on www.c rguide.org and in several publications [20,[22][23][24]. The traditional approach consisted of the following steps: Primary analyst: Write notes during interview and "code" into matrix immediately after interview; use comments and highlight areas that need clari cation or timestamps.…”