Process evaluation of complex interventions: Medical Research Council guidance

Abstract

Process evaluation is an essential part of designing and testing complex interventions. New MRC guidance provides a framework for conducting and reporting process evaluation studiesAttempts to tackle problems such as smoking and obesity increasingly use complex interventions. These are commonly defined as interventions that comprise multiple interacting components, although additional dimensions of complexity include the difficulty of their implementation and the number of organisational levels they target. 1 Randomised controlled trials are regarded as the gold standard for establishing the effectiveness of interventions, when randomisation is feasible. However, effect sizes do not provide policy makers with information on how an intervention might be replicated in their specific context, or whether trial outcomes will be reproduced. Earlier MRC guidance for evaluating complex interventions focused on randomised trials, making no mention of process evaluation. 2 Updated guidance recognised the value of process evaluation within trials, stating that it "can be used to assess fidelity and quality of implementation, clarify causal mechanisms and identify contextual factors associated with variation in outcomes." 3 However, it did not provide guidance for carrying out process evaluation. Developing guidance for process evaluationIn 2010, a workshop funded by the MRC Population Health Science Research Network discussed the need for guidance on process evaluation. 4 There was consensus that researchers, funders, and reviewers would benefit from guidance. A group of researchers with experience and expertise in evaluating complex interventions was assembled to produce the guidance. In line with the principles followed in developing earlier MRC guidance documents, draft guidance was produced drawing on literature reviews, process evaluation case studies, workshops, and discussions at conferences and seminars. It was then circulated to academic, policy, and practice stakeholders for comment. Around 30 stakeholders provided written comments on the draft structure, while others commented during conference workshops run throughout the development process. A full draft was recirculated for further review, before being revised and approved by key MRC funding panels.Although the aim was to provide guidance on process evaluation of public health interventions, the guidance is highly relevant to complex intervention research in other domains, such as health services and education. The full guidance (www.populationhealthsciences.org/Process-Evaluation-Guidance.html) begins by setting out the need for process evaluation. It then presents a review of influential theories and frameworks which informed its development, before offering practical recommendations, and six detailed case studies. In this article, we provide an overview of the new framework and summarise our practical recommendations using one of the case studies as an example. MRC process evaluation frameworkThe new framework builds on the process evaluation themes descri...

Key strategies to improve systems for managing patient complaints within health facilities – what can we learn from the existing literature?

“…First, limited understanding of contexts of effective interventions . There is a growing recognition of the importance of the context in ensuring the success of complex health systems interventions [41,117–122], including improvements to patient complaints systems. While specific components of the interventions are usually well described, it is often unclear which contextual facilitators and barriers affected their effectiveness, and how.…”

Psychiatric service staff perceptions of implementing a shared decision-making tool: a process evaluation study

“…As in all implementation of new methods, manager’s attitudes and level of commitment is crucial (Aarons et al, 2016; Gifford et al, 2013; Moore et al, 2015). Managers need to provide sustainable structures, with a clear vision of the new method as constituting a formal part of the ongoing activities.…”

“…The framework for process evaluation from Moore et al (2015) was applied to support the planning and designing of the project. The framework has been beneficial for the design of the study and the data collection strategy.…”

“…In designing the process evaluation of the intervention, we developed a blueprint guided by Moore et al (2015) which included three evaluation components: contextual factors, implementation issues and mechanisms of impact (Table III). …”

“…Seven major codes emerged in the inductive content analysis of group interview transcriptions: (1) Perceptions of limited added value of the DST, (2) Limited embrace of SDM, (3) Barriers associated with service users, (4) Barriers associated with workflow, (5) Intervention-related problems (6) Perceptions of added value with the DST. In the next step, a directed approach followed (Patton, 2002), which involved placing the inductively derived codes into categories based on the process evaluation components from Moore et al (2015). The idea of employing a deductive method that is derived from an existing model, is that a well-validated model based on a perspective relevant for the study may serve to clarify the results.…”