Measuring sustainability of a grassroots program in a large integrated health care delivery system: the Warrior to Soul Mate Program

Stolldorf, Deonni P.; Fortune-Britt, Alice G.; Nieuwsma, Jason A.; Gierisch, Jennifer M.; Datta, Santanu K.; Angel, Clyde T.; Millspaugh, Dick; Jackson, George L.

doi:10.3138/jmvfh.2017-0007

Cited by 7 publications

(17 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, most teams sustained their practice, unless they were not able to maintain necessary staffing or resources. Two CFIR constructs, Engaging Key Stakeholders and Available Resources, were key for both initial implementation success and sustainment, which is similar to factors noted in prior literature on programs both within and outside the VHA [37,38]. Other CFIR constructs did not have a major influence on implementation and sustainment outcomes due to the high level of implementation support provided, which helped teams overcome most barriers.…”

Section: Discussionsupporting

confidence: 67%

Implementation and sustainment of diverse practices in a large integrated health system: a mixed methods study

Nevedal

Reardon

Jackson

et al. 2020

Implement Sci Commun

Self Cite

View full text Add to dashboard Cite

Background One goal of health systems seeking to evolve into learning health systems is to accelerate the implementation and sustainment of evidence-based practices (EBPs). As part of this evolution, the Veterans Health Administration (VHA) developed the Innovation Ecosystem, which includes the Diffusion of Excellence (DoE), a program that identifies and diffuses Gold Status Practices (GSPs) across facilities. The DoE hosts an annual “Shark Tank” competition in which leaders bid on the opportunity to implement a GSP with 6 months of implementation support. Over 750 diverse practices were submitted in cohorts 2 and 3 of Shark Tank; 23 were designated GSPs and were implemented in 31 VA networks or facilities. As part of a national evaluation of the DoE, we identified factors contributing to GSP implementation and sustainment. Methods Our sequential mixed methods evaluation of cohorts 2 and 3 of Shark Tank included semi-structured interviews with at least one representative from 30/31 implementing teams ( N = 78/105 people invited) and survey responses from 29/31 teams ( N = 39/47 invited). Interviews focused on factors influencing implementation and future sustainment. Surveys focused on sustainment 1.5–2 years after implementation. The Consolidated Framework for Implementation Research (CFIR) informed data collection and directed content analysis. Ordinal scales were developed inductively to rank implementation and sustainment outcomes. Results Over 50% of teams (17/30) successfully implemented their GSP within the 6-month implementation period. Despite extensive implementation support, significant barriers related to centralized decision-making, staffing, and resources led to partial ( n = 6) or no ( n = 7) implementation for the remaining teams. While 12/17 initially successful implementation teams reported sustained use of their GSP, over half of the initially unsuccessful teams ( n = 7/13) also reported sustained GSP use 1.5 years after the initial implementation period. When asked at 6 months, 18/27 teams with complete data accurately anticipated their future sustainability based on reported sustainment an average of 1.5 years later. Conclusions Most teams implemented within 6 months and/or sustained their GSP 1.5 years later. High levels of implementation and sustainment across diverse practices and teams suggest that VHA’s DoE is a successful large-scale model of diffusion. Team predictions about sustainability after the first 6 months of implementation provide a promising early assessment and point of intervention to increase sustainability.

show abstract

Section: Discussionsupporting

confidence: 67%

Implementation and sustainment of diverse practices in a large integrated health system: a mixed methods study

Nevedal

Reardon

Jackson

et al. 2020

Implement Sci Commun

Self Cite

View full text Add to dashboard Cite

show abstract

“…One quarter (25%; n = 16) used a mixed-methods design [ 50 – 65 ], followed by 14% ( n = 9) studies that used a multi-methods design [ 66 – 74 ]. Only 12% ( n = 8) of the included studies used a quantitative research design [ 75 – 82 ]. Of the 31 qualitative studies, 94% ( n = 29) used interviews to collect evaluation data [ 19 – 22 , 24 – 40 , 42 – 49 ]; however, only 59% ( n = 17) of those reported interviews as the sole data collection method [ 20 , 21 , 24 – 27 , 29 , 30 , 32 , 33 , 37 , 38 , 40 , 43 , 45 , 47 , 48 ].…”

Section: Resultsmentioning

confidence: 99%

“…Surveys or questionnaires accompanied the interviews in 22% ( n = 2) of studies [ 69 , 73 ]. Of the nine quantitative studies, 100% ( n = 8) administered a survey or questionnaire to collect sustainability evaluation data [ 75 – 82 ] Table 4…”

Section: Resultsmentioning

confidence: 99%

Identifying existing approaches used to evaluate the sustainability of evidence-based interventions in healthcare: an integrative review

et al. 2022

View full text Add to dashboard Cite

Background There is limited evidence to evaluate the sustainability of evidence-based interventions (EBIs) for healthcare improvement. Through an integrative review, we aimed to identify approaches to evaluate the sustainability of evidence-based interventions (EBIs) and sustainability outcomes. Methods Following Whittemore and Knafl’s methodological process: (1) problem identification; (2) literature search; (3) data evaluation; (4) data analysis; and (5) presentation, a comprehensive search strategy was applied across five databases. Included studies were not restricted by research design; and had to evaluate the sustainability of an EBI in a healthcare context. We assessed the methodological quality of studies using the Mixed Methods Appraisal Tool. Results Of 18,783 articles retrieved, 64 fit the inclusion criteria. Qualitative designs were most commonly used for evaluation (48%), with individual interviews as the predominant data collection method. Timing of data collection varied widely with post-intervention data collection most frequent (89%). Of the 64 studies, 44% used a framework, 26% used a model, 11% used a tool, 5% used an instrument, and 14% used theory as their primary approach to evaluate sustainability. Most studies (77%) did not measure sustainability outcomes, rather these studies focused on sustainability determinants. Discussion It is unclear which approach/approaches are most effective for evaluating sustainability and what measures and outcomes are most commonly used. There is a disconnect between evaluating the factors that may shape sustainability and the outcomes approaches employed to measure sustainability. Our review offers methodological recommendations for sustainability evaluation research and highlights the importance in understanding mechanisms of sustainability to advance the field.

show abstract

“…For example, the EBP is perceived to have low sustainability due to inadequate funding or lack of priority. Operationally, the goal is to determine whether the conditions indicative of sustaining EBPs are in place, and if not, to guide efforts to put such conditions into place [3,4].…”

Section: Background Evaluating Sustainment Of Evidence-based Practices Is Challengingmentioning

confidence: 99%

Sustainment of Diverse Evidence-Based Practices Disseminated in the Veterans Health Administration (VHA): Development and Administration of a Pragmatic Instrument

Reardon¹,

Nevedal

Widerquist

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Background There are challenges associated with measuring sustainment of evidence-based practices (EBPs). First, the terms sustainability and sustainment are often falsely conflated: sustainability assesses the likelihood of an EBP being in use in the future while sustainment assesses the extent to which an EBP is (or is not) in use. Second, grant funding often ends before sustainment can be assessed. The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) is one of few large-scale models of diffusion; it seeks to identify and disseminate practices across the VHA system. The DoE sponsors “Shark Tank” competitions, in which leaders bid on the opportunity to implement a practice with 6 months of implementation support. As part of an ongoing evaluation of the DoE, we sought to develop and administer a pragmatic instrument to assess sustainment of DoE practices.Methods In June 2020, surveys were sent to 64 facilities that were part of the DoE evaluation. We began analysis by comparing alignment of quantitative and qualitative responses; some facility representatives reported in the open text box of the survey that their practice was on a temporary hold due to COVID-19 but answered the primary outcome question differently. As a result, the team reclassified the primary outcome of these facilities to Sustained: Temporary COVID-Hold. Following this reclassification, the number and percent of facilities in each category was calculated. We used directed content analysis, guided by the Consolidated Framework for Implementation Research (CFIR), to analyze open text box responses. Results A representative from forty-one facilities (64%) completed the survey. Among responding facilities, 29/41 sustained their practice, 1/41 partially sustained their practice, 8/41 had not sustained their practice, and 3/41 had never implemented their practice. Sustainment rates increased between Cohorts 1 – 4. Conclusions The development and administration of our pragmatic survey allowed us to assess sustainment of DoE practices. Planned updates to the survey will enable flexibility in assessing sustainment and its determinants at any phase after adoption. This assessment approach can flex with the longitudinal and dynamic nature of sustainment, including capturing nuances in outcomes when practices are on a temporary hold.

show abstract

Measuring sustainability of a grassroots program in a large integrated health care delivery system: the Warrior to Soul Mate Program

Cited by 7 publications

References 33 publications

Implementation and sustainment of diverse practices in a large integrated health system: a mixed methods study

Implementation and sustainment of diverse practices in a large integrated health system: a mixed methods study

Identifying existing approaches used to evaluate the sustainability of evidence-based interventions in healthcare: an integrative review

Sustainment of Diverse Evidence-Based Practices Disseminated in the Veterans Health Administration (VHA): Development and Administration of a Pragmatic Instrument

Contact Info

Product

Resources

About