The Use and Misuse of Counterfactuals in Ethical Machine Learning

Abstract

The use of counterfactuals for considerations of algorithmic fairness and explainability is gaining prominence within the machine learning community and industry. This paper argues for more caution with the use of counterfactuals when the facts to be considered are social categories such as race or gender. We review a broad body of papers from philosophy and social sciences on social ontology and the semantics of counterfactuals, and we conclude that the counterfactual approach in machine learning fairness and social explainability can require an incoherent theory of what social categories are. Our findings suggest that most often the social categories may not admit counterfactual manipulation, and hence may not appropriately satisfy the demands for evaluating the truth or falsity of counterfactuals. This is important because the widespread use of counterfactuals in machine learning can lead to misleading results when applied in high-stakes domains. Accordingly, we argue that even though counterfactuals play an essential part in some causal inferences, their use for questions of algorithmic fairness and social explanations can create more problems than they resolve. Our positive result is a set of tenets about using counterfactuals for fairness and explanations in machine learning. CCS CONCEPTS • Computing methodologies → Philosophical/theoretical foundations of artificial intelligence; Machine learning; • Social and professional topics → Socio-technical systems; Race and ethnicity.

Fairness and Data Protection Impact Assessments

“…Unfortunately, this fairness metric also comes with some difficulties in analyzing and evaluating the counterfactual statements. See Kasirzadeh and Smart [26] for some principled arguments against the prevalent use of counterfactual fairness in social contexts.…”

Promises and Challenges of Causality for Ethical Machine Learning

“…There are also concerns about considering categories such as race or gender as a cause [30,26,15,22]. From one perspective, most of these attributes are determined at the time of an individual's conception and are modeled as source nodes in a causal graph which can directly or indirectly influence the descendent variables.…”

“…Alternatively, many view attributes such as race or gender as social constructs that evolve over the course an individual's life. Recently, [15,22] studied epistemological and ontological aspects of counterfactuals in the context of fairness evaluation. In [15], the authors argue that social categories such as race may not admit counterfactual manipulation.…”

How Does Counterfactually Augmented Data Impact Models for Social Computing Constructs?

“…One suggested solution to address the issue of spurious features is counterfactually augmented data (CAD)-instances generated by human annotators that are minimally edited to flip their labeland their variations such as iterative benchmark design (Potts et al, 2020), contrast data generation (Gardner et al, 2020), 1 and their combination . Drawing on the rich history of counterfactuals (Pearl, 2018;Lewis, 2013;Kasirzadeh and Smart, 2021), the promise of CAD is to offer a causality-based framework where only cues that are meaningfully associated with the construct are edited -which is expected to be conducive to models learning less spurious features. Indeed, recent work has shown that models trained on CAD generalize better out of domain (Kaushik et al, 2020;Samory et al, 2021).…”