The Effect of Intensive Implementation Support on Fidelity for Four Evidence-Based Psychosis Treatments: A Cluster Randomized Trial

Abstract

Purpose Service providers need effective strategies to implement evidence-based practices (EBPs) with high fidelity. This study aimed to evaluate an intensive implementation support strategy to increase fidelity to EBP standards in treatment of patients with psychosis. Methods The study used a cluster randomized design with pairwise assignment of practices within each of 39 Norwegian mental health clinics. Each site chose two of four practices for implementation: physical health care, antipsychotic medication management, family psychoeducation, illness management and recovery. One practice was assigned to the experimental condition (toolkits, clinical training, implementation facilitation, data-based feedback) and the other to the control condition (manual only). The outcome measure was fidelity to the EBP, measured at baseline and after 6, 12, and 18 months, analyzed using linear mixed models and effect sizes. Results The increase in fidelity scores (within a range 1–5) from baseline to 18 months was significantly greater for experimental sites than for control sites for the combined four practices, with mean difference in change of 0.86 with 95% CI (0.21; 1.50), p = 0.009). Effect sizes for increase in group difference of mean fidelity scores were 2.24 for illness management and recovery, 0.68 for physical health care, 0.71 for antipsychotic medication management, and 0.27 for family psychoeducation. Most improvements occurred during the first 12 months. Conclusions Intensive implementation strategies (toolkits, clinical training, implementation facilitation, data-based feedback) over 12 months can facilitate the implementation of EBPs for psychosis treatment. The approach may be more effective for some practices than for others.

Implementation of Guidelines on Family Involvement for Persons with Psychotic Disorders (IFIP): A Cluster Randomised Controlled Trial

“…The progressive nature of the FPE model probably explains the gradual increase in scores in the experimental arm. The success rate is good compared to previous studies, which report mean scores of 3.30-4.00 and 39-50% of sites reaching adequate fidelity after 18-24 months of implementation support (Bond et al, 2009a, b;Kealey et al, 2015;Ruud et al, 2021). However, these studies experienced high rates of discontinuation or unsuccessful implementation.…”

“…A qualitative exploration of the implementation process, in terms of barriers and facilitators, has also been published (Hansson et al, 2022). The ISP was based on the seminal work of the National Evidence-Based Practices (NEBP) project (Bond et al, 2009a;McHugo et al, 2007), and on a recent Norwegian RCT (Ruud et al, 2021). We adopted elements such as the constitution of a local implementation team, regular fidelity measurements with tailored feedback and on-site supervision, kick-off sessions, training and supervision Fig.…”

“…Fidelity measurements may also enable researchers to distinguish between failure of the intervention and failure of implementation . Previous fidelitybased studies on the implementation of FPE have been either experimental non-randomised trials (Kealey et al, 2015;McHugo et al, 2007), or unable to demonstrate significant increases in fidelity (Ruud et al, 2021).…”

Can Systematic Implementation Support Improve Program Fidelity by Improving Care Providers’ Perceptions of Implementation Factors? A Cluster Randomized Trial

“…Each unit was assigned to receive implementation support for one of the two practices they had chosen and not for the other. Earlier investigations from this trial have shown that the implementation support intervention had a signi cant positive impact on the delity scores [21].…”

Implementation strategies for telemental health: a systematic review

“…This could be addressed by future research, for example, cluster randomised controlled trials to formally compare the effectiveness of different implementation strategies linked to specific outcomes, although it would also be helpful to evaluate a theoretically informed approach to selecting a bundle of implementation strategies to fit a particular context, using more robust study designs. Inclusion of a control group in future evaluations of the implementation of TMH is critical: without this we will remain unable to establish causal links between the presence or absence of a strategy to support implementation, or bundle thereof, and success of implementation (38) offers an illustration of a clustered randomised evaluation of different interventions to support the implementation of four evidenced-based psychosis treatments. Within this study, what was randomised was not the clinical therapies, but rather the level of implementation support, which was limited in the control arm (provision of treatment manuals) and substantially enhanced in the intervention arm to include toolkits, training, implementation facilitation, and data-based feedback.…”