The use of Qualitative Comparative Analysis (QCA) to address causality in complex systems: a systematic review of research on public health interventions

Abstract

Background Qualitative Comparative Analysis (QCA) is a method for identifying the configurations of conditions that lead to specific outcomes. Given its potential for providing evidence of causality in complex systems, QCA is increasingly used in evaluative research to examine the uptake or impacts of public health interventions. We map this emerging field, assessing the strengths and weaknesses of QCA approaches identified in published studies, and identify implications for future research and reporting. Methods PubMed, Scopus and Web of Science were systematically searched for peer-reviewed studies published in English up to December 2019 that had used QCA methods to identify the conditions associated with the uptake and/or effectiveness of interventions for public health. Data relating to the interventions studied (settings/level of intervention/populations), methods (type of QCA, case level, source of data, other methods used) and reported strengths and weaknesses of QCA were extracted and synthesised narratively. Results The search identified 1384 papers, of which 27 (describing 26 studies) met the inclusion criteria. Interventions evaluated ranged across: nutrition/obesity (n = 8); physical activity (n = 4); health inequalities (n = 3); mental health (n = 2); community engagement (n = 3); chronic condition management (n = 3); vaccine adoption or implementation (n = 2); programme implementation (n = 3); breastfeeding (n = 2), and general population health (n = 1). The majority of studies (n = 24) were of interventions solely or predominantly in high income countries. Key strengths reported were that QCA provides a method for addressing causal complexity; and that it provides a systematic approach for understanding the mechanisms at work in implementation across contexts. Weaknesses reported related to data availability limitations, especially on ineffective interventions. The majority of papers demonstrated good knowledge of cases, and justification of case selection, but other criteria of methodological quality were less comprehensively met. Conclusion QCA is a promising approach for addressing the role of context in complex interventions, and for identifying causal configurations of conditions that predict implementation and/or outcomes when there is sufficiently detailed understanding of a series of comparable cases. As the use of QCA in evaluative health research increases, there may be a need to develop advice for public health researchers and journals on minimum criteria for quality and reporting.

Predicting animal to human translation: A proof of concept study using qualitative comparative analysis

“…We initiated these analyses as a first exploration and proof of principle of the QCA-method in meta-research of animal-to-human translation. QCAs have successfully been performed on data from systematic literature reviews in other fields [16, 17, 23, 24]. However, to the best of our knowledge, we are the first to perform a QCA with animal metadata, and to use it to analyse animal-to-human translation.…”

“…It is based on set theory and Boolean algebra. QCA is increasingly used to identify specific configurations of factors predicting an outcome in other fields [16, 17]. We reanalysed the data from our preceding review with a crisp-set QCA (csQCA) [18].…”

Qualitative comparative analysis of the implementation fidelity of a workplace sedentary reduction intervention

“…It could also be that who delivers the intervention, and their quality are more likely to lead to employee changes than simply adhering to the protocol. An additional key strength is the use of QCA to provide an innovative and systematic approach for understanding implementation outcomes across contexts and its association with intervention effectiveness (39). Another strength is the inclusion of a diverse set of worksites allowing for greater generalizability of intervention delity implications.…”

Are realist randomised controlled trials possible? A reflection on the INCLUSIVE evaluation of a whole-school, bullying-prevention intervention

“…Some publications have explicitly referred to realist evaluation [ 11 , 13 , 14 ] while others have not. Likewise, some of these methods, such as moderator and mediator analyses [ 15 , 16 ], are controversial within realist circles, while others are not [ 17 – 19 ].…”

“…The use of statistical moderation analyses in particular reflects a recognition that any conjunctions are contingent on other factors. The use of QCA is also possible within trials, as we have demonstrated, and this rests on an assumption that causality is best assessed by exploring the contingent inter-relationships between multiple factors [ 19 ]. Questions about causal attribution are central to trial analyses not because interventions are thought to be the exclusive, determining source of causation but because trials seek to explore how the mechanisms triggered by the introduction of new resources into contexts interacts with all the other mechanisms operating in that context to generate new outcomes.…”