Learning Gender-Neutral Word Embeddings

Zhao, Jieyu; Zhou, Yichao; Li, Zeyu; Wang, Wei; Chang, Kai-Wei

doi:10.48550/arxiv.1809.01496

Cited by 51 publications

(60 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Artificial intelligence systems are known not only to perpetuate social biases, but they may also amplify existing cultural assumptions and inequalities [2]. While most work on biases in word embeddings focuses on a single social category (e.g., gender, race) [1,3,4,5,6], the lack of work on identifying intersectional biases, the bias associated with populations defined by multiple categories [7], leads to an incomplete measurement of social biases [8,9]. For example, Caliskan et al's Word Embedding Association Test (WEAT) quantifies biases documented by the validated psychological methodology of the Implicit Association Test (IAT) [10].…”

Section: Arxiv:200603955v2 [Cscy] 22 Jun 2020mentioning

confidence: 99%

Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases

Guo

Çalışkan

2021

Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society

102

View full text Add to dashboard Cite

With the starting point that implicit human biases are reflected in the statistical regularities of language, it is possible to measure biases in static word embeddings [1]. With recent advances in natural language processing, state-of-the-art neural language models generate dynamic word embeddings dependent on the context in which the word appears. Current methods of measuring social and intersectional biases in these contextualized word embeddings rely on the effect magnitudes of bias in a small set of pre-defined sentence templates. We propose a new comprehensive method, Contextualized Embedding Association Test (CEAT), based on the distribution of 10,000 pooled effect magnitudes of bias in potential embedding variations and a random-effects model, dispensing with templates. Experiments on social and intersectional biases show that CEAT finds evidence of all tested biases and provides comprehensive information on the variability of effect magnitudes of the same bias in different contexts. Furthermore, we develop two methods, Intersectional Bias Detection (IBD) and Emergent Intersectional Bias Detection (EIBD), to automatically identify the intersectional biases and emergent intersectional biases from static word embeddings in addition to measuring them in contextualized word embeddings. We present the first algorithmic bias detection findings on how intersectional group members are associated with unique emergent biases that do not overlap with the biases of their constituent minority identities. IBD achieves an accuracy of 81.6% and 82.7%, respectively, when detecting the intersectional biases of African American females and Mexican American females. EIBD reaches an accuracy of 84.7% and 65.3%, respectively, when detecting the emergent intersectional biases unique to African American females and Mexican American females. The probability of random correct identification in these tasks ranges from 12.2% to 25.5% in IBD and from 1.0% to 25.5% in EIBD.

show abstract

Section: Arxiv:200603955v2 [Cscy] 22 Jun 2020mentioning

confidence: 99%

Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases

Guo

Çalışkan

2021

Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society

102

View full text Add to dashboard Cite

show abstract

“…There are several methods to modify the embedding structure in ways that mitigate the encoded bias. While there are more complicated optimization-based ones designed for specific tasks in gender bias in text [67], we describe a subset of four debiasing methods that are quite simple to actuate (although nuances of them may be confusing), and rely specifically on the concept subspaces identified earlier. Again, VERB serves as the perfect visual medium to explain these debiasing methods.…”

Section: Bias Mitigation Methodsmentioning

confidence: 99%

“…There are several intrinsic [7,14] and extrinsic [12,67] measures to determine how much bias is contained by word embeddings. When bias is removed [5,12,13], these measures help determine how effective the bias removal has been.…”

Section: Bias Evaluation Methodsmentioning

confidence: 99%

“…As a result, over the last five years, there has been a rapid development of many debiasing mechanisms to pinpoint, quantify, and mitigate the biases within these representations. While some have focused on correcting for these biases outside of the embeddings or in spite of them [67], we focus on approaches that directly analyze and mitigate issues in these embedded representations. First, these representations are widespread and so the techniques are highly transferable.…”

Section: Debiasing Embedded Representationsmentioning

confidence: 99%

See 1 more Smart Citation

VERB: Visualizing and Interpreting Bias Mitigation Techniques for Word Representations

Rathore¹,

Dev²,

Phillips³

et al. 2021

Preprint

View full text Add to dashboard Cite

Fig. 1. With VERB, users can explore the high-dimensional word representations interactively before, during, and after applying bias mitigation techniques. (A) Embedding View highlights a subset of word embeddings using dimensionality reduction and visualizes the step-by-step transformations of their embeddings across various debiasing techniques. (B) Control Panel enables users to configure each debiasing technique and provides controls to iterate through each step of the transformation. (C) Explanation Panel gives a step-by-step description of the transformation. In this example, in the Embedding View, word representations are reoriented with the concept direction (Gender) along the x-axis. They are then transformed by a debiasing technique called the Linear Projection that removes this concept. These transformed representations are visualized again in (D) after a new dimensionality reduction, where there is no longer a well-defined gender concept, and hence the stereotypical associations are mitigated.

show abstract

“…This work has raised important questions regarding the impact of these embedded biases on downstream decision-making, given the increasing usage of these models in various applications. Consequently, much work has also been dedicated to creating standardized diagnostic tests to detect these biases (Caliskan et al, 2017;May et al, 2019;Nadeem et al, 2020;Sweeney and Najafian, 2019) and to remove them (Bolukbasi et al, 2016;Zhao et al, 2018;Manzini et al, 2019), although the extent to which this is possible is still under debate (Gonen and Goldberg, 2019). In fact, research has found that "The biases found in Internet-scale language models like GPT-2 are representative of the data on which the model was trained" (So-laiman et al, 2019), which can be directly linked to the presence of hate speech on the Internet (Abid et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus

Luccioni,

Viviano

2021

Preprint

View full text Add to dashboard Cite

Whereas much of the success of the current generation of neural language models has been driven by increasingly large training corpora, relatively little research has been dedicated to analyzing these massive sources of textual data. In this exploratory analysis, we delve deeper into the Common Crawl, a colossal web corpus that is extensively used for training language models. We find that it contains a significant amount of undesirable content, including hate speech and sexually explicit content, even after filtering procedures. We discuss the potential impacts of this content on language models and conclude with future research directions and a more mindful approach to corpus collection and analysis.

show abstract

Learning Gender-Neutral Word Embeddings

Cited by 51 publications

References 19 publications

Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases

Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases

VERB: Visualizing and Interpreting Bias Mitigation Techniques for Word Representations

What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus

Contact Info

Product

Resources

About