Diversity in sociotechnical machine learning systems

Abstract

There has been a surge of recent interest in sociocultural diversity in machine learning research. Currently, however, there is a gap between discussions of measures and benefits of diversity in machine learning, on the one hand, and the broader research on the underlying concepts of diversity and the precise mechanisms of its functional benefits, on the other. This gap is problematic because diversity is not a monolithic concept. Rather, different concepts of diversity are based on distinct rationales that should inform how we measure diversity in a given context. Similarly, the lack of specificity about the precise mechanisms underpinning diversity’s potential benefits can result in uninformative generalities, invalid experimental designs, and illicit interpretations of findings. In this work, we draw on research in philosophy, psychology, and social and organizational sciences to make three contributions: First, we introduce a taxonomy of different diversity concepts from philosophy of science, and explicate the distinct epistemic and political rationales underlying these concepts. Second, we provide an overview of mechanisms by which diversity can benefit group performance. Third, we situate these taxonomies of concepts and mechanisms in the lifecycle of sociotechnical machine learning systems and make a case for their usefulness in fair and accountable machine learning. We do so by illustrating how they clarify the discourse around diversity in the context of machine learning systems, promote the formulation of more precise research questions about diversity’s impact, and provide conceptual tools to further advance research and practice.

Interdisciplinarity, Gender Diversity, and Network Structure Predict the Centrality of AI Organizations

“…especially in scientific research [56]. Beyond optimizing team performance, increasing gender diversity, or, more broadly, participation of marginalized groups in AI field is a crucial step towards AI systems that do not perpetuate existing patterns of societal injustice [11,29,32,40,42,77]. Also in line with our hypotheses, we find that organizations with high betweenness centrality authors in AI coauthorship networks tend to be more interdisciplinary.…”

“…Diversity refers to differences between individuals on any attributes (e.g., demographic characteristics) that may lead to the perception that another person is different from the self [69,71]. The lack of gender and race diversity in the AI sector has been identified as a factor contributing to the rise of AI systems that replicate societal inequities and that offer poor service to marginalized populations [32,40,77]. Diversity also plays an important role in organizational network structures, as increased diversity in organizations has been found to improve decision making through increased creativity and innovation [3,28,53].…”

In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd Workers

“…Similarly, majority voting or weighted voting is often used to aggregate worker labels for consensus (Hung et al, 2013 ; Sheshadri and Lease, 2013 ). However, we also know that when workers have consistent, systematic group biases, the aggregation will serve to reinforce and amplify the group bias rather than mitigate it (Ipeirotis et al, 2010 ; Sen et al, 2015 ; Dumitrache et al, 2018 ; Fazelpour and De-Arteaga, 2022 ).…”

“…We describe our own experiences with this in section 5.1.1. Annotators, on the other hand, also bring with them their own variety of implicit biases which the requester may not detect or understand (Ipeirotis et al, 2010 ; Sen et al, 2015 ; Dumitrache et al, 2018 ; Geva et al, 2019 ; Al Kuwatly et al, 2020 ; Fazelpour and De-Arteaga, 2022 ).…”

Does AI-Assisted Fact-Checking Disproportionately Benefit Majority Groups Online?

“…We ask whether algorithms used to assist factcheckers tend to identify falsehoods that impact majority groups while allowing misinformation that impacts minority groups to proliferate relatively unimpeded. Additionally, we investigate whether explicitly incorporating notions of diversity (as discussed in Fazelpour and De-Arteaga (2022)) into the training data and AI-assisted fact-checking workflow improves overall outcomes and affects the distribution of benefits from fact-checking.…”