Diversity in Sociotechnical Machine Learning Systems

Abstract

There has been a surge of recent interest in sociocultural diversity in machine learning (ML) research, with researchers (i) examining the bene ts of diversity as an organizational solution for alleviating problems with algorithmic bias, and (ii) proposing measures and methods for implementing diversity as a design desideratum in the construction of predictive algorithms. Currently, however, there is a gap between discussions of measures and bene ts of diversity in ML, on the one hand, and the broader research on the underlying concepts of diversity and the precise mechanisms of its functional bene ts, on the other. This gap is problematic because diversity is not a monolithic concept. Rather, di erent concepts of diversity are based on distinct rationales that should inform how we measure diversity in a given context. Similarly, the lack of speci city about the precise mechanisms underpinning diversity's potential bene ts can result in uninformative generalities, invalid experimental designs, and illicit interpretations of ndings. In this work, we draw on research in philosophy, psychology, and social and organizational sciences to make three contributions: First, we introduce a taxonomy of di erent diversity concepts from philosophy of science, and explicate the distinct epistemic and political rationales underlying these concepts. Second, we provide an overview of mechanisms by which diversity can bene t group performance. Third, we situate these taxonomies-of concepts and mechanisms-in the lifecycle of sociotechnical ML systems and make a case for their usefulness in fair and accountable ML. We do so by illustrating how they clarify the discourse around diversity in the context of ML systems, promote the formulation of more precise research questions about diversity's impact, and provide conceptual tools to further advance research and practice.

Diversity in sociotechnical machine learning systems

“…For example, enhanced participation and contestation can improve group decision-making in a way that can be pertinent to considerations ranging from operationalization and measurement Martin Jr et al 2020) to human-AI teaming (Kompa et al 2021;Raghu et al 2019) in the AI lifecycle. But, they can also result in delays and even undermine group cohesion and decision quality (Dobbe et al 2020;Fazelpour and De-Arteaga 2021). Similarly, when thinking about issues of compositionality, we are likely to face the so-called diversity-stability trade-off familiar from other complex systems, where too much variability-a requirement for complementarity-can destabilize the system and degrade performance (Eliassi-Rad et al 2020;Page 2019).…”

Data Representativeness in Accessibility Datasets: A Meta-Analysis

“…Our overarching goal lies in understanding the current state of representativeness of marginalized groups in AI datasets (along the axes of age, gender, and race & ethnicity) with a specific focus on disabled data contributors. This is relevant to the greater discourse around AI, ethics, and fairness, as marginalized communities tend to be under-represented in data [47], perpetuating cycles of exclusion as technology advances even for technologies that meant to promote inclusion such as assistive technology. We contribute to this important ongoing discussion through our analysis of 190 accessibility datasets.…”

“…More so, AI research has adopted diversity considerations deeply in the ongoing challenge of responsible and ethical AI [24,42,113]. Much conversation has been associated with the concepts around balanced representation of subgroups (e.g., equal participation of racial sub-groups within a focal group) [47]. A growing number of studies have explored bias and performance disparities of AI systems concerning representation [38,108], especially influenced by demographic attributes like age [36,97,124], gender [18,83,142,162], race [18,96], socioeconomic status [34], and disability status [56,179].…”

“…Prior work investigating issues associated with diversity in AI datasets has mostly focused on examining differences in model performance across pre-defined demographic attributes to draw implications for diversity [18,34,162]. This often leaves inquiries about the benefits and appropriate implementation of diversity in data unanswered [47], except for a few exceptions (as shown in Table 1) that explicitly analyzed datasets or issues related to datasets in terms of demographic representation like gender and other sociocultural attributes (e.g., language) to explore the root causes of bias and misrepresentation. These studies concluded that such AI datasets (often image datasets) are skewed towards certain demographics, uncovering under-representation of older adults [109,128], darker-skin, and females [109,185], and lack of geographical diversity [148].…”

Justice in Misinformation Detection Systems: An Analysis of Algorithms, Stakeholders, and Potential Harms

“…Or does this group have access to unique knowledge that gives them more expertise on a given topic than a general fact-checker? Some labelers may have epistemically advantaged standpoints [29], and aggregation mechanisms such as majority voting may fail to represent their assessments [17].…”