What works for whom in which circumstances? On the need to move beyond the ‘what works?’ question in organizational intervention research

Abstract

A debate has arisen out of the need to understand true intervention outcomes in the social sciences. Traditionally, the randomized, controlled trial (RCT) that answers the question of 'what works' has been considered the gold standard. Although RCTs have been favoured in organizational intervention research, there has been an increasing interest in understanding the influence of context and intervention processes on the outcomes of such interventions. In the present critical essay, we question the suitability of RCTs and meta-analyses to evaluate the effectiveness of organizational interventions and we suggest that realist evaluation that seeks to answer the questions of what works for whom in which circumstances may present a more suitable framework. We argue that examining the content and process mechanisms through which organizational interventions are effective, and the conditions under which these are triggered, will enable us to better understand how interventions achieve the desired outcomes of improved employee health and well-being. We suggest that organizational intervention content and process mechanisms may help bring about the desired outcomes of improved employee health and well-being and that contextual factors determine whether these mechanisms are triggered. KeywordsCMO-configurations, context-mechanism-outcome configurations, critical essay, metaanalysis, organizational interventions, randomized controlled trial, realist evaluation, realist synthesis What works for whom?What works for whom?The randomized, controlled trial has been perceived as the gold standard for evaluating interventions (Guyatt et al., 1995). The RCT approach employs a successionist approach to causation, i.e. that randomization holds the context constant, there are no differences at baseline between the intervention and the control group, and outcomes can be inferred from comparing those exposed to the intervention to those not exposed to the intervention. When intended outcomes can be observed statistically above and beyond outcomes in the control group, it is assumed that these outcomes can be attributed to the intervention. Implicit in this line of thinking is that intervention outcomes can be aggregated What works for whom?In the field of organizational interventions the RCT design has been met with criticism (Nielsen, 2013a). Organizational interventions can be defined as planned, behavioural, theory-based actions that aim to improve employee health and well-being through changing the way work is designed, organized and managed (e.g. Nielsen, 2013a;Richardson and Rothstein, 2008). In the present essay, we argue that realist evaluation (Pawson, 2013;Pawson and Tilley, 1997) may offer an opportunity to develop an integrated context, process and outcome evaluation framework that may advance our theoretical understanding of which elements of organizational interventions may be effective and in which conditions we can expect positive outcomes. Realist evaluation offers a way to conduct rigorous, theory-based analyse...

Well-being and HRM in the changing workplace

“…However, it can also present a challenge for HR professionals in terms of knowing how best to manage the 'remote worker' or virtual teams, for example, one issue arises in relation to whether processes should be measured or if it is more relevant to measure performance of these individuals by outputs. Nielsen and Miraglia's (2017) notion of what works and for whom and when, particularly resonates here. Joyce et al (2010) also raise the issue that there are limited cost-benefits analyses of wellbeing interventions which can present an additional challenge if not measured as how can organisations then determine whether investing in employee well-being is indeed a win-win or 'mutual gain' or whether it can lead to conflicting outcomes?…”

“…Nielsen & Miraglia, 2017). Briner and Walshe (2015) propose that identifying the well-being problem is the first step to identifying which intervention may be most appropriate, and this may vary across organisations.…”

“…Interventions research, and evaluation of interventions, are imperative if organisations are to understand the value of investing in employee well-being and perceiving this as 'mutual gains' as opposed to 'conflicting outcomes' . As Nielsen and Miraglia (2017) question in their recent paper, 'when, and for whom' do interventions work? It is clear from this recent paper, and from previous papers on interventions that 'one size does not fit all' and that if contextual factors are not taken into account, this can potentially have adverse consequences for the effectiveness of any intervention.…”

How might patient involvement in healthcare quality improvement efforts work—A realist literature review

“…We, therefore, conclude that the results add to existing knowledge and can guide stakeholders in health-care organizations and microsystems. 32,71,76 Furthermore, studying organizations in other sectors that have successfully involved users would enable a deeper understanding of how health-care organizations can involve patients in QI efforts ever more successfully.…”

Building and sustaining work engagement – a participatory action intervention to increase work engagement in nursing staff

“…This study also highlights the need for multi-wave, longitudinal intervention designs, with their superior ability to assess the causal relationships between variables, explore the psychological mechanisms underlying an intervention, and test theory (Nielsen & Miraglia, 2017). Such studies are imperative for mediation analyses, with cross-lagged and daily diary studies offering some ways that future studies could assess longitudinal relationships.…”