Between complexity and generalization: Addressing evaluation challenges with QCA

Abstract

This article argues that Qualitative Comparative Analysis can be a useful method in case-based evaluations for two reasons: a) it is aimed at causal inference and explanation, leading to theory development; b) it is strong on external validity and generalization, allowing for theory testing and refinement. After a brief introduction to QCA, the specific type of causality handled by QCA is discussed. QCA is shown to offer improvements over Mill's methods by handling asymmetric and multiple-conjunctural causality in addition to counterfactual reasoning. It thereby allows the explicitly separate analysis of necessity and sufficiency, recognizing the relevance of causal packages as well as single causes and of multiple causal paths leading to the same outcome (equifinality). It is argued that QCA can generalize findings to a small, medium and large number of cases.

Process Tracing and Contribution Analysis: A Combined Approach to Generative Causal Inference for Impact Evaluation

“…And further, it is expected that the actions of the intervention are an essential part of the causal package, in which case the intervention can claim to have made a difference -not by itself, but within the causal package 2 -and as such is making a causal claim (Mayne 2012a). This is not the place for a detailed discussion of contributory causes which are neither necessary nor sufficient to bring about an outcome (see Befani 2013 andMackie 1974 for a discussion of INUS (Insufficient but Necessary part of an…”

The effect of leadership and other contextual conditions on the ecological and socio-economic success of small-scale fisheries in Southeast Asia

“…QCA extends John Stuart Mill's long-standing approaches to identifying single-cause attribution of outcomes (Befani, 2013). QCA was developed as a tool to analyze causal relationships between a set of conditions and an outcome (Schneider and Wagemann, 2012) and has served as an important bridging methodology between small-n case-based research and large-n statistical analyses.…”

Navigating the Complexities of Qualitative Comparative Analysis

“…First, the number of cases has often been cited to rationalize the choice of QCA in preference to other techniques of empirical data analysis. I prove this justification to rest on misconceptions about the comparative advantages of different methods and show why case numbers are likewise Balthasar (2006) a Applied Administration Evaluation use mvQCA Befani (2013) Agenda Methodological -- Befani et al (2007) Agenda Methodological -- Blackman (2008) Applied irrelevant for determining the variant of QCA. Second, I argue for a radical rethink of current policy concerning the treatment of necessity relations.…”

“…To date, at least 19 publications can be identified in peer-reviewed English-language journals. In the group of ''agenda articles,' ' Befani (2013), Befani, Ledermann, and Sager (2007), Blackman, Wistow, and Byrne (2013), Sager and Andereggen (2012), Verweij and Gerrits (2013), and Warren, Wistow, and Bambra (2014) showcase the usefulness of QCA for addressing various research questions evaluators face. In contrast, ''applied articles'' focus on empirical substance and include Balthasar (2006), Blackman (2008), Chatterley et al (2014), Dy et al (2005), Ford, Duncan, and Ginter (2005), Glatman-Freedman et al (2010), Gross and Garvin (2011), Holvoet and Dewachter (2013), Kahwati et al (2011), Lam and Ostrom (2010), Ledermann (2012), Ozegowski (2013), and Thygeson et al (2012).…”