Examining Gender Bias in Languages with Grammatical Gender

Abstract

Recent studies have shown that word embeddings exhibit gender bias inherited from the training corpora. However, most studies to date have focused on quantifying and mitigating such bias only in English. These analyses cannot be directly extended to languages that exhibit morphological agreement on gender, such as Spanish and French. In this paper, we propose new metrics for evaluating gender bias in word embeddings of these languages and further demonstrate evidence of gender bias in bilingual embeddings which align these languages with English. Finally, we extend an existing approach to mitigate gender bias in word embeddings under both monolingual and bilingual settings. Experiments on modified Word Embedding Association Test, word similarity, word translation, and word pair translation tasks show that the proposed approaches effectively reduce the gender bias while preserving the utility of the embeddings.

The Woman Worked as a Babysitter: On Biases in Language Generation

“…Through an annotated dataset, we address the appropriateness of sentiment scores as a proxy for measuring bias 1 are generated text with occupation contexts, and the latter two are generated text with respect contexts. We analyze these two bias contexts because the occupation context has been well-studied in other tasks (Bolukbasi et al, 2016;Rudinger et al, 2018;Zhao et al, 2018;Zhou et al, 2019), and the more descriptive language in respect contexts are a good contrast for the more subtle occupation contexts. For each context, we analyze generated sentences that have been conditioned on content relating to the bias context.…”

Multi-Dimensional Gender Bias Classification

“…Gender affects myriad aspects of NLP, including corpora, tasks, algorithms, and systems Costa-jussà, 2019;Sun et al, 2019). For example, statistical gender biases are rampant in word embeddings (Jurgens et al, 2012;Bolukbasi et al, 2016;Caliskan et al, 2017;Garg et al, 2018;Zhao et al, 2018b;Basta et al, 2019;Chaloner and Maldonado, 2019;Du et al, 2019;Ethayarajh et al, 2019;Kaneko and Bollegala, 2019;Kurita et al, 2019;-including multilingual ones (Escudé Font and Costa-jussà, 2019;Zhou et al, 2019)-and affect a wide range of downstream tasks including coreference resolution (Zhao et al, 2018a;Cao and Daumé III, 2020;Emami et al, 2019), part-ofspeech and dependency parsing (Garimella et al, 2019), language modeling (Qian et al, 2019;Nangia et al, 2020), appropriate turn-taking classification (Lepp, 2019), relation extraction (Gaut et al, 2020), identification of offensive content (Sharifirad and Matwin, 2019;, and machine translation (Stanovsky et al, 2019;Hovy et al, 2020).…”

Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation

“…Particular attention has been paid to uncovering, analyzing, and removing gender biases in word embeddings (Basta et al, 2019;Kaneko and Bollegala, 2019;Zhao et al, , 2018bBolukbasi et al, 2016). This word embedding work has even extended to multilingual work on gender-marking Williams et al, 2019;Zhou et al, 2019;. Despite these efforts, many methods for debiasing embeddings have only succeeded in hiding word embedding biases as opposed to removing them -making gender debiasing still an open area of research.…”