Developing a framework for evaluation: a Theory of Change for complex workplace mental health interventions

Abstract

Background There is a gap between the necessity of effective mental health interventions in the workplace and the availability of evidence-based information on how to evaluate them. The available evidence outlines that mental health interventions should follow integrated approaches combining multiple components related to different levels of change. However, there is a lack of robust studies on how to evaluate multicomponent workplace interventions which target a variety of outcomes at different levels taking into account the influence of different implementation contexts. Method We use the MENTUPP project as a research context to develop a theory-driven approach to facilitate the evaluation of complex mental health interventions in occupational settings and to provide a comprehensive rationale of how these types of interventions are expected to achieve change. We used a participatory approach to develop a ToC involving a large number of the project team representing multiple academic backgrounds exploiting in tandem the knowledge from six systematic reviews and results from a survey among practitioners and academic experts in the field of mental health in SMEs. Results The ToC revealed four long-term outcomes that we assume MENTUPP can achieve in the workplace: 1) improved mental wellbeing and reduced burnout, 2) reduced mental illness, 3) reduced mental illness-related stigma, and 4) reduced productivity losses. They are assumed to be reached through six proximate and four intermediate outcomes according to a specific chronological order. The intervention consists of 23 components that were chosen based on specific rationales to achieve change on four levels (employee, team, leader, and organization). Conclusions The ToC map provides a theory of how MENTUPP is expected to achieve its anticipated long-term outcomes through intermediate and proximate outcomes assessing alongside contextual factors which will facilitate the testing of hypotheses. Moreover, it allows for a structured approach to informing the future selection of outcomes and related evaluation measures in either subsequent iterations of complex interventions or other similarly structured programs. Hence, the resulting ToC can be employed by future research as an example for the development of a theoretical framework to evaluate complex mental health interventions in the workplace.

Implementation and evaluation of a multi-level mental health promotion intervention for the workplace (MENTUPP): study protocol for a cluster randomised controlled trial

“…Furthermore, the evaluation strategy has been refined to reflect the core research questions and reduce the number of questions, which will improve participant experience. Moreover, a comprehensive ToC has been developed that describes in detail how we assume the intervention to work, under which conditions and what kind of changes we expect to see on the different organisational levels in the short, intermediate and long term [ 66 ].…”

“…Quantitative and qualitative measures will be utilised to collect all necessary data for outcome evaluation, process evaluation, and economic evaluation of the MENTUPP intervention. Measures were selected based on the Theory of Change (ToC) expected as a result of the intervention [ 66 ]. An overview of assessment is presented in Fig.…”

“…This is a common practice and does not affect the validity or reliability of the scale. The items need to be rated on 5-point Likert scale X X X X X BAT: burnout [ 66 ] The BAT consists of 12 items and measures 4 dimensions of burnout: exhaustion, mental distance, emotional impairment, cognitive impairment. Contrary to the Maslach Burnout inventory it measures cognitive malfunctioning and produces a single burnout score that is easy to interpret.…”

The effects of different types of organisational workplace mental health interventions on mental health and wellbeing in healthcare workers: a systematic review

“…MENTUPP focusses on SMEs in three sectors: healthcare, information and communications technology (ICT) and construction ( https://www.mentuppproject.eu/publications/ ). Based on a theory of change (Tsantila et al 2023 ), the MENTUPP intervention aims to promote wellbeing, reduce both clinical (depressive, anxiety disorders) and nonclinical (stress, burnout) mental health issues and the stigma of mental (ill-) health (Arensman et al 2023 ). A review about organisational level mental health interventions in the construction industry has been published (Greiner et al 2022 ) and a publication of a review about these type of interventions in ICT workers is currently in preparation.This systematic review was performed in order to synthesize the effectiveness of organisational interventions on mental health outcomes in healthcare workers.…”

Implementing a complex mental health intervention in occupational settings: process evaluation of the MENTUPP pilot study

“… 14 Hence, we developed a theory of change (ToC) that maps out how our intervention leads to its desired outcomes and under what conditions, allowing us to select appropriate outcome indicators. 15 In addition, we conducted a comprehensive process evaluation of the pilot study to inform us about the implementation context and its impact allowing us to optimise our intervention, implementation strategy, evaluation approach and ToC ahead of the full-scale evaluation. The optimised MENTUPP intervention and the revised ToC will be published elsewhere in the future after the completion of the cRCT.…”