Conservative solutions for progress: on solution types when combining QCA with in-depth Process-Tracing

Abstract

What is the most appropriate QCA solution type when engaging in a multimethod design that includes QCA and in-depth process-tracing (PT)? While either the intermediate or the parsimonious solution are generally favored in QCA-only studies, we identify important challenges that can emerge when selecting those solutions in a QCA-PT multimethod study. We particularly highlight the risk of mechanistic heterogeneity, omitted conditions, and draw the attention on the issue of generalization. We discuss each of these intertwined challenges in depth, and explain why the conservative solution is useful to consider in addressing them. We substantiate our arguments by drawing on a recently completed evaluation study that was commissioned by the Flemish ESF Agency in Belgium. In the study, we combined QCA and theory-guided in-depth process-tracing to uncover under what combinations of conditions (QCA) a training programme would lead to successful training transfer and how (PT) this happened in the successful cases. The article highlights the need to carefully consider the selection of solution types in any multimethod design comprising QCA.

Identifying diverse paths toward successful policy performance in Organization for Economic Co‐operation and Development (OECD) and European Union countries: A Qualitative Comparative Analysis (QCA) exploitation of the Sustainable Governance Indicators (SGI) data

“…As an illustration, we will refer to the enhanced intermediate solution, which is typically recommended for interpretation purposes. In a nutshell, the argument is that such an intermediate solution is better informed by theory and by some substantive case knowledge, which makes it both theoretically and empirically “safer” than the parsimonious solution (see Schneider & Wagemann, 2012 and Álamos‐Concha et al, 2021). Other solutions can be found in the Appendix 5 .…”

“…Fifth, a combination of QCA and process tracing via a multi‐method research strategy (Álamos‐Concha et al, 2021; Rihoux et al, 2021) would bring three further potential enrichments. To start with, it would enable us to examine both what worked (core combinations of conditions, via QCA) and how it worked (causal mechanisms, via process tracing), thereby not only explaining variation in policy performance but also understanding how the process toward successful policy performance actually unfolded (unpacking the “black box” of the process).…”

The Identification of Causal Mechanisms in Sustainable Urban Transitions—A Systematic Approach to Case Selection

“…The causal mechanism is defined as a system in which the activities of the actors transfer causal forces from the causes to the outcomes [11,24,26,36,37]. The mechanistic perspective of causation seeks to uncover the causal process [18], to explain how and through what kind of process an outcome is brought about or produced [38]. The causal mechanism is not an intervening variable, but a theoretical statement or conceptual construct on how a set of variables, or the processes linked to such variables, cause the empirical outcome [39].…”

“…Process tracing is a within-case method that uncovers the causal mechanisms through a systematic study of the process that links a cause, or a conjunction of causes, with an outcome [24,[39][40][41]; the mechanisms that enable the outcome to actually unfold [23,25,36,38,42]. As we have commented above, the causes identified at the cross-case level make the outcomes possible; if the agents finally achieve the outcome, that is explained by a within-case analysis via process tracing [35].…”

Choosing a qualitative comparative analysis solution in multi-method impact evaluation

“…Often, QCA is used jointly with case studies (Schneider and Rohlfing, 2016; Thomann and Maggetti, 2020). Specifically, the sequencing of QCA with process tracing to identify mechanisms behind the configuration leading to the effect is gaining popularity (Álamos-Concha et al, 2021; Beach, 2018; Beach and Rohlfing, 2018; Schneider and Rohlfing, 2013). For example, Álamos-Concha et al (2020) investigated configurations of conditions leading to the effective transfer of training.…”