Quality of intervention delivery in a cluster randomised controlled trial: a qualitative observational study with lessons for fidelity

Abstract

BackgroundUnderstanding intervention fidelity is an essential part of the evaluation of complex interventions because fidelity not only affects the validity of trial findings, but also because studies of fidelity can be used to identify barriers and facilitators to successful implementation, and so provide important information about factors likely to impact the uptake of the intervention into clinical practice. Participant observation methods have been identified as being particularly valuable in studies of fidelity, yet are rarely used. This study aimed to use these methods to explore the quality of implementation of a complex intervention (Safewards) on mental health wards during a cluster randomised controlled trial. Specific aims were firstly to describe the different ways in which the intervention was implemented, and secondly to explore the contextual factors moderating the quality of intervention delivery, in order to inform ‘real world’ implementation of the intervention.MethodsSafewards was implemented on 16 mental health wards in England. We used Research Assistants (RAs) trained in participant observation to record qualitative observational data on the quality of intervention delivery (n = 565 observations). At the end of the trial, two focus groups were conducted with RAs, which were used to develop the coding framework. Data were analysed using thematic analysis.ResultsThere was substantial variation in intervention delivery between wards. We observed modifications to the intervention which were both fidelity consistent and inconsistent, and could enhance or dilute the intervention effects. We used these data to develop a typology which describes the different ways in which the intervention was delivered. This typology could be used as a tool to collect qualitative observational data about fidelity during trials. Moderators of Safewards implementation included systemic, interpersonal, and individual factors and patient responses to the intervention.ConclusionsOur study demonstrates how, with appropriate training in participant observation, RAs can collect high-quality observational data about the quality of intervention delivery during a trial, giving a more complete picture of ‘fidelity’ than measurements of adherence alone.Trial registrationISRCTN registry; IRSCTN38001825. Registered 29 August 2012Electronic supplementary materialThe online version of this article (doi:10.1186/s13063-017-2189-8) contains supplementary material, which is available to authorized users.

Systematic review of interventions to improve constant observation on adult inpatient psychiatric wards

“…Interventions were delivered by training members of staff (Bisconer, ; Bowers et al ., ; Dennis, ; James et al ., ; Kettles & Paterson, ; Reynolds et al ., ), meeting clinical teams (Moran, ) and getting approval by senior clinical staff (Ray, Perkins & Meijer, ; Ray et al ., ). The remaining studies either did not describe how interventions were delivered (Bowers et al ., , ; Carr, ; Sullivan et al ., ; Triplett et al ., ) or described a change that was already taking place at the time of the study (Jenkins, Dye & Foy, ).…”

“…Only the 16 studies that were formally evaluated were rated for quality using the EPHPP tool. One intervention had a strong quality rating (Bowers et al ., ; James et al ., ) and two interventions had a moderate quality rating (Jenkins, Dye & Foy, ; Ray et al ., ). The remaining interventions were all rated as poor based on their study design (see appendix Table S3).…”

“…A total of 13 interventions in this review had been formally evaluated by 16 studies. Two interventions were described by more than one study (Bowers et al ., ; James et al ., ; Ray, Perkins & Meijer, ; Ray et al ., ), and one intervention was replicated twice in separate settings (Bowers et al ., , ). Eight out of the 13 interventions were specifically designed to address the constant observation practice.…”

“…Larger interventions were also reviewed, providing that they included attempts to improve or reduce constant observation. These interventions also attempted to reduce violence and aggression (Bisconer, ; Sullivan et al ., ), reduce all harmful incidents on the ward (Bowers et al ., , , ; James et al ., ) or reduce other forms of restrictive practice (Bowers et al ., , ; James et al ., ; Sullivan et al ., ).…”

“…Most studies gathered information directly from clinical wards by speaking with staff or patients (Bisconer, ; Bowers et al ., , ; Carr, ; Dennis, ; Sullivan et al ., ; Triplett et al ., ), observing the practice (Bowers et al ., , ) or reviewing ward data (Carr, ; Dennis, ). Other studies also looked at existing research to develop their intervention (Bowers et al ., , , ; Carr, ; Dodds & Bowles ; James et al ., ; Ray, Perkins & Meijer, ; Ray et al ., ). The remaining studies designed their interventions by reviewing existing clinical guidelines (Kettles & Paterson, ; Reynolds et al ., ) and using author's personal experience (Moran, ).…”