Predicting the Long-Term Sustainability of Evidence-Based Practices in Mental Health Care: An 8-Year Longitudinal Analysis

Abstract

Few studies have examined predictors of long-term sustainability of evidence-based practices in mental health. This study used assessments of five evidence-based practices implemented in 49 sites in eight states at baseline and years 2, 4, and 8. Program characteristics, implementation characteristics, reinforcement activities, and sustainability factors were used to predict program survival status. The majority of predictors were not significant. Supervisor turnover in year 4 predicted survival status in year 8, but site characteristics, fidelity at implementation, quality improvement activities, and post-implementation activities had little impact on long-term program survival. This study extends previous sustainability research by examining the long-term impact of internal program factors over a substantial period of time using longitudinal prediction. Future research should also consider the influence of external factors such as financial policies.

The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care

“…Many studies have used a qualitative or mixed-methods approach to understand barriers and facilitators to sustainability (5, 10, 37). Consistent with organizational contexts in community settings (e.g., schools), many of these studies, particularly those in clinics or hospitals, suggest that funding, organizational factors and support (e.g., champions, supervision), and practitioner/workforce characteristics (e.g., turnover) are particularly influential (13,98,125 …”

“…It is also important that researchers specify the unit of data collection and analysis for both sustainability determinants and outcomes. Prospective, multilevel, and mixed-methods study designs are ideal for studying sustainability (94,98). For example, quantitative surveys could be used to examine predictors of sustainability in multiple waves over time in conjunction with measures of sustainability outcomes, whereas qualitative surveys (e.g., in-depth case studies, interviews, ethnographic research) could be conducted among stakeholders at multiple levels to contextualize the findings within specific organizational settings, to provide insight into why programs were or were not sustained, and to understand differential processes that occurred across organizations (13,98,131,147).…”

Predicting evidence-based treatment sustainment: results from a longitudinal study of the Adolescent-Community Reinforcement Approach

“…However, their study was limited in that external contextual factors were not examined. Peterson et al [16] also examined the inner context or setting factors related to the sustainment of five mental health EBTs over an 8-year period. Unlike our study, they did not find implementation quality predictive of long-term sustainment; however, like our study, staff turnover appeared important.…”

“…For example, the use of small samples are common (e.g., prospective data from 32 programs [14]; 11 systems operating in two states [15]), studies that have been limited to the examination of demonstration projects (e.g., [10]) rather than programs operated by established organizations, and some studies that have only examined a subset of hypothesized factors, such as inner context characteristics (e.g., [5, 16]) and few rigorous studies have been conducted that use longitudinal data and measure the quality or extent of EBT sustainment. Therefore, a continued need exists for comprehensive empirical studies assessing the extent or quality of sustainment and the numerous hypothesized factors that predict it.…”

The what, when, and why of implementation frameworks for evidence-based practices in child welfare and child mental health service systems

“…Most researchers and community stakeholders recognize the need for ongoing monitoring and evaluation throughout implementation to promote sustained use of EBPs with fidelity (e.g., Herschell, 2010; Meyers et al, 2012). Relatedly, studies highlight the importance of sustainability planning during the initial phases of implementation to increase the likelihood of implementation success (Beidas, Mehta, Atkins, Solomon, & Merz, 2013; Peterson et al, 2014; Pluye, Potvin, & Denis, 2004; Shediac-Rizkallah & Bone; 1998; Wiltsey-Stirman et al, 2012). …”