On Measuring and Mitigating Biased Inferences of Word Embeddings

Abstract

Word embeddings carry stereotypical connotations from the text they are trained on, which can lead to invalid inferences in downstream models that rely on them. We use this observation to design a mechanism for measuring stereotypes using the task of natural language inference. We demonstrate a reduction in invalid inferences via bias mitigation strategies on static word embeddings (GloVe). Further, we show that for gender bias, these techniques extend to contextualized embeddings when applied selectively only to the static components of contextualized embeddings (ELMo, BERT).

UNQOVERing Stereotyping Biases via Underspecified Questions

“…Arguably this setting is more natural, as it better aligns with how systems are used in real life. Several notable examples are coreference resolution (Rudinger et al, 2018;Zhao et al, 2018;Kurita et al, 2019), machine translation (Stanovsky et al, 2019;Cho et al, 2019), textual entailment (Dev et al, 2020a), language generation (Sheng et al, 2019), or clinical classification (Zhang et al, 2020).…”

“…Such studies on model bias have led to many bias mitigation techniques (e.g., Bolukbasi et al, 2016b;Dev et al, 2020a;Ravfogel et al, 2020;Dev et al, 2020b). In this work, we focus on exploring biases across QA models and expect that our framework could also help future efforts on bias mitigation.…”

“…We define templates (T ) for all four bias classes, and select common names, nationalities, ethnicities, and religions for our subject list (X). We use the occupations from Dev et al (2020a) and statements that capture prejudices from StereoSet (Nadeem et al, 2020) to create our attribute list (A). Table 1 shows the sizes of slot-fillers in our templates and the resulted data sizes.…”

“…Unfortunately, these representations learn stereotypes often enmeshed in the massive body of text used to train them (Sun et al, 2019). These biases are subsequently passed on to downstream tasks such as co-reference resolution (Rudinger et al, 2018;Zhao et al, 2018), textual entailment (Dev et al, 2020a), and translation (Stanovsky et al, 2019). Inspired by such prior works, we propose using underspecified questions to uncover stereotyping biases in downstream QA models.…”

“…Note that prior approaches have often focused on discovering biases by recognizing when a model is categorically incorrect (Stanovsky et al, 2019;Dev et al, 2020a;Nadeem et al, 2020). Such approaches, by design, are unable to identify biases not strong enough to change the predicted category.…”