Bias in Bios

De‐Arteaga, Maria; Romanov, Alexey; Wallach, Hanna; Chayes, Jennifer; Borgs, Christian; Chouldechova, Alexandra; Geyik, Sahin Cem; Kenthapadi, Krishnaram; Kalai, Adam Tauman

doi:10.1145/3287560.3287572

Cited by 153 publications

(83 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For example, in unsupervised learning, word embeddings often contain biases (Bolukbasi et al, 2016;Caliskan et al, 2017;Garg et al, 2018) which persist even after attempts to remove them (Gonen and Goldberg, 2019). There are many examples of bias in supervised learning contexts: YouTube's captioning models make more errors when transcribing women (Tatman, 2017), AAE is more likely to be misclassified as non-English by widely used language classifiers (Blodgett and O'Connor, 2017), numerous gender and racial biases exist in sentiment classification systems (Kiritchenko and Mohammad, 2018), and errors in both co-reference resolution systems and occupational classification models reflect gendered occupational patterns (Zhao et al, 2018;De-Arteaga et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

“…Recent work has shown evidence of substantial bias in machine learning systems, which is typically a result of bias in the training data. This includes both supervised (Blodgett and O'Connor, 2017;Tatman, 2017;Kiritchenko and Mohammad, 2018;De-Arteaga et al, 2019) and unsupervised natural language processing systems (Bolukbasi et al, 2016;Caliskan et al, 2017;Garg et al, 2018). Machine learning models are currently being deployed in the field to detect hate speech and abusive language on social media platforms including Facebook, Instagram, and Youtube.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Racial Bias in Hate Speech and Abusive Language Detection Datasets

Davidson¹,

Bhattacharya²,

Weber³

2019

Proceedings of the Third Workshop on Abusive Language Online

319

261

View full text Add to dashboard Cite

Technologies for abusive language detection are being developed and applied with little consideration of their potential biases. We examine racial bias in five different sets of Twitter data annotated for hate speech and abusive language. We train classifiers on these datasets and compare the predictions of these classifiers on tweets written in African-American English with those written in Standard American English. The results show evidence of systematic racial bias in all datasets, as classifiers trained on them tend to predict that tweets written in African-American English are abusive at substantially higher rates. If these abusive language detection systems are used in the field they will therefore have a disproportionate negative impact on African-American social media users. Consequently, these systems may discriminate against the groups who are often the targets of the abuse we are trying to detect.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Racial Bias in Hate Speech and Abusive Language Detection Datasets

Davidson¹,

Bhattacharya²,

Weber³

2019

Proceedings of the Third Workshop on Abusive Language Online

319

261

View full text Add to dashboard Cite

show abstract

“…For example, the sentence he is an engineer is more likely to appear in a corpus than she is an engineer due to the current gender disparity in engineering. Consequently, any NLP system that is trained such a corpus will likely learn to associate engineer with men, but not with women (De-Arteaga et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology

Zmigrod¹,

Mielke²,

Wallach³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Self Cite

157

122

View full text Add to dashboard Cite

Gender stereotypes are manifest in most of the world's languages and are consequently propagated or amplified by NLP systems. Although research has focused on mitigating gender stereotypes in English, the approaches that are commonly employed produce ungrammatical sentences in morphologically rich languages. We present a novel approach for converting between masculine-inflected and feminineinflected sentences in such languages. For Spanish and Hebrew, our approach achieves F 1 scores of 82% and 73% at the level of tags and accuracies of 90% and 87% at the level of forms. By evaluating our approach using four different languages, we show that, on average, it reduces gender stereotyping by a factor of 2.5 without any sacrifice to grammaticality.

show abstract

“…That is, artifacts like predictive accuracy are referred to as being "allocated" across people qua the group of which they are a member. 10 For example, researchers describe "fair allocation of predictions" across groups of people (Zliobaite, 2015) and refer to "allocational harms" that are understood as disparities in a model's accuracy for different groups of people (De-Arteaga et al, 2019;Davidson et al, 2019). In another example, a model's predictive accuracy is explicitly characterized as a "resource" to be fairly allocated via a distributive rule (Hashimoto et al, 2018).…”

Section: What Are Resources?mentioning

confidence: 99%

Measuring justice in machine learning

Lundgard

2020

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

View full text Add to dashboard Cite

How can we build more just machine learning systems? To answer this question, we need to know both what justice is and how to tell whether one system is more or less just than another. That is, we need both a definition and a measure of justice. Theories of distributive justice hold that justice can be measured (in part) in terms of the fair distribution of benefits and burdens across people in society. Recently, the field known as fair machine learning has turned to John Rawls's theory of distributive justice for inspiration and operationalization. However, philosophers known as capability theorists have long argued that Rawls's theory uses the wrong measure of justice, thereby encoding biases against people with disabilities. If these theorists are right, is it possible to operationalize Rawls's theory in machine learning systems without also encoding its biases? In this paper, I draw on examples from fair machine learning to suggest that the answer to this question is no: the capability theorists' arguments against Rawls's theory carry over into machine learning systems. But capability theorists don't only argue that Rawls's theory uses the wrong measure, they also offer an alternative measure. Which measure of justice is right? And has fair machine learning been using the wrong one?

show abstract

Bias in Bios

Cited by 153 publications

References 20 publications

Racial Bias in Hate Speech and Abusive Language Detection Datasets

Racial Bias in Hate Speech and Abusive Language Detection Datasets

Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology

Measuring justice in machine learning

Contact Info

Product

Resources

About