Lipstick on a Pig:

Abstract

Word embeddings are widely used in NLP for a vast range of tasks. It was shown that word embeddings derived from text corpora reflect gender biases in society. This phenomenon is pervasive and consistent across different word embedding models, causing serious concern. Several recent works tackle this problem, and propose methods for significantly reducing this gender bias in word embeddings, demonstrating convincing results. However, we argue that this removal is superficial. While the bias is indeed substantially reduced according to the provided bias definition, the actual effect is mostly hiding the bias, not removing it. The gender bias information is still reflected in the distances between "gender-neutralized" words in the debiased embeddings, and can be recovered from them. We present a series of experiments to support this claim, for two debiasing methods. We conclude that existing bias removal techniques are insufficient, and should not be trusted for providing gender-neutral modeling.

Going Beyond T-SNE: Exposing whatlies in Text Embeddings

“…Figure 7 shows the result of applying the debiasing technique, highlighting that some of the spurious correlations have indeed been removed. It is important to note though, that the above technique does not reliably remove all relevant bias in the embeddings and that bias is still measurably existing in the embedding space as Gonen and Goldberg (2019) have shown. This can be verified with whatlies, by plotting the neighbours of the biased and debiased space: As the output shows, the neighbourhoods of maid in the biased and debiased space are almost equivalent, with e.g.…”

Perturbation Sensitivity Analysis to Detect Unintended Model Biases

“…Fairness research in NLP has seen tremendous growth in the past few years (e.g., (Bolukbasi et al, 2016;Caliskan et al, 2017;Webster et al, 2018;Díaz et al, 2018;Dixon et al, 2018;De-Arteaga et al, 2019;Gonen and Goldberg, 2019;Manzini et al, 2019)) over a range of NLP tasks such as co-reference resolution and machine translation, as well as the tasks we studied -sentiment analysis and toxicity prediction. Some of this work study bias by swapping names in sentence templates (Caliskan et al, 2017;Kiritchenko and Mohammad, 2018;May et al, 2019;Gonen and Goldberg, 2019); however they use synthetic sentence templates, while we extract naturally occurring sentences from the target corpus.…”

Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation

“…Under their evaluation, they find they can nearly perfectly remove bias in an analogical reasoning task. However, subsequent work (Gonen and Goldberg, 2019;Hall Maudslay et al, 2019) has indicated that gender bias still lingers in the embeddings, despite Bolukbasi et al (2016)'s strong experimental results. In the development of their method, Bolukbasi et al (2016) make a critical and unstated assumption: Gender bias forms a linear subspace of word embedding space.…”

“…As previously noted, there are now multiple bias removal methodologies (Zhao et al, 2018(Zhao et al, , 2019May et al, 2019) that have succeed the method by Bolukbasi et al (2016). Furthermore Gonen and Goldberg (2019) point out multiple flaws in Bolukbasi et al (2016)'s bias mitigation technique and the aforementioned methods. Nonetheless we believe that this method has received sufficient attention from the community such that research into its properties is both interesting and useful.…”

“…First, we consider the Word Embedding Association Test (WEAT; Caliskan et al, 2017); we notice that across five non-linear kernels and convex combinations thereof, there is seemingly no significant difference between the non-linear bias mitigation technique and the linear one. Secondly, we consider the professions task (Bolukbasi et al, 2016;Gonen and Goldberg, 2019) that measures how word embeddings representing different professions are potentially genderstereotyped. Again, as with the WEAT evaluation, we find that our non-linear bias mitigation technique performs on par with the linear method.…”