Development of a measure of model fidelity for mental health Crisis Resolution Teams

Abstract

BackgroundCrisis Resolution Teams (CRTs) provide short-term intensive home treatment to people experiencing mental health crisis. Trial evidence suggests CRTs can be effective at reducing hospital admissions and increasing satisfaction with acute care. When scaled up to national level however, CRT implementation and outcomes have been variable. We aimed to develop and test a fidelity scale to assess adherence to a model of best practice for CRTs, based on best available evidence.MethodsA concept mapping process was used to develop a CRT fidelity scale. Participants (n = 68) from a range of stakeholder groups prioritised and grouped statements (n = 72) about important components of the CRT model, generated from a literature review, national survey and qualitative interviews. These data were analysed using Ariadne software and the resultant cluster solution informed item selection for a CRT fidelity scale. Operational criteria and scoring anchor points were developed for each item. The CORE CRT fidelity scale was then piloted in 75 CRTs in the UK to assess the range of scores achieved and feasibility for use in a 1-day fidelity review process. Trained reviewers (n = 16) rated CRT service fidelity in a vignette exercise to test the scale’s inter-rater reliability.ResultsThere were high levels of agreement within and between stakeholder groups regarding the most important components of the CRT model. A 39-item measure of CRT model fidelity was developed. Piloting indicated that the scale was feasible for use to assess CRT model fidelity and had good face validity. The wide range of item scores and total scores across CRT services in the pilot demonstrate the measure can distinguish lower and higher fidelity services. Moderately good inter-rater reliability was found, with an estimated correlation between individual ratings of 0.65 (95% CI: 0.54 to 0.76).ConclusionsThe CORE CRT Fidelity Scale has been developed through a rigorous and systematic process. Promising initial testing indicates its value in assessing adherence to a model of CRT best practice and to support service improvement monitoring and planning. Further research is required to establish its psychometric properties and international applicability.Electronic supplementary materialThe online version of this article (doi:10.1186/s12888-016-1139-4) contains supplementary material, which is available to authorized users.

Crisis resolution and home treatment in the UK: A survey of model fidelity using a novel review methodology

“…This was a comprehensive assessment with a robust fidelity review process. The CRT Fidelity Scale was the result of a rigorous development process that produced a valid measure; and the review process included reviewers with different backgrounds (service users, carers, clinicians and researchers), creating multi‐perspective reviewing teams (Lloyd‐Evans et al, ). The scale was able to discriminate between services on all the main subscales, with both individual item scores and overall total scores showing a large range, and demonstrating that all items were attainable.…”

“…For practical reasons, an extended vignette was constructed from anonymized examples of review paperwork for use in the exercise, rather than data from in vivo reviews. As a result, the extent to which these results reflect the inherent reliability of the scale is still unclear (Lloyd‐Evans et al, ).…”

“…Scoring is completed by a team of three reviewers (for the purposes of this survey, the teams included one clinician, one service user or carer, and one researcher), following a one‐day audit of services. The measure has demonstrated acceptable inter‐rater reliability in testing with vignettes (correlation between individual ratings = 0.65 (95% CI 0.54–0.76); intra‐class correlation between averaged ratings across raters = 0.97 (95% CI 0.95–0.98); Lloyd‐Evans et al ). A total score of 39–116 is considered poor fidelity to the model, a total score of 117–155 is considered moderate fidelity to the model, and a score of 156–195 is considered good fidelity to the model.…”

Hospital length of stay among older people treated with flexible and integrative psychiatric service models in Germany

“…A study on the fidelity of implementation of new integrative and continuous care models using the Index of Fidelity of ACT in 18 programmes showed that higher scores on the fidelity scale were moderately to strongly correlated with reductions in hospital length of stay . Also, CRT with a higher fidelity were more successful in reducing hospital admissions . A British study reported that the AHLS was significantly shorter for all patients referred to crisis services than for those in the combined comparison group over 2 years' follow‐up .…”

Home Treatment: Von der Forschung zur klinischen Umsetzung

“…Bei der Implementierung in die Routineversorgung sollte die Modelltreue beachtet werden, da sie für den Erfolg des Behandlungsmodells entscheidend ist [19]. Hierzu werden auch Instrumente zur Evaluation der Modelltreue beitragen, um sowohl strukturelle als auch inhaltliche Kriterien für Home Treatment-Angebote besser zu definieren [17].…”