Measuring Bias in Contextualized Word Representations

Abstract

Contextual word embeddings such as BERT have achieved state of the art performance in numerous NLP tasks. Since they are optimized to capture the statistical properties of training data, they tend to pick up on and amplify social stereotypes present in the data as well. In this study, we (1) propose a template-based method to quantify bias in BERT;(2) show that this method obtains more consistent results in capturing social biases than the traditional cosine based method; and (3) conduct a case study, evaluating gender bias in a downstream task of Gender Pronoun Resolution. Although our case study focuses on gender bias, the proposed technique is generalizable to unveiling other biases, including in multiclass settings, such as racial and religious biases.

Entity-Centric Contextual Affective Analysis

“…To the best of our knowledge, this is the first work to target affective dimensions in pre-trained contextualized word embeddings. Our findings are consistent with prior work suggesting that contextualized embeddings capture biases from training data (Zhao et al, 2019;Kurita et al, 2019) and that these models perform best when trained on in-domain data (Alsentzer et al, 2019).…”

Hurtful words

“…In this work, we examine the demographic parity, equality of opportunity for the positive class, and equality of opportunity for the negative class. 1 First, we demonstrate that there are significant differences in the log probability bias scores [36] of clinical text for different genders. These scores examine the probability of filling in the gender demographics given medical context.…”

“…debiasing) word embeddings. While individual works study how contextual word embeddings capture biases [5,36,62], to date the creation of debiasing methods has been limited to non-contextual word embeddings models (e.g. GLoVe [49], Word2Vec [45]).…”

“…We use log probability bias scores, a previously proposed method for evaluating evaluating biases in contextual language models [36], which measures prior-adjusted likelihoods of predicting a given word for a fill-in-the-blanks task (Algorithm 1). For each topic, a set of template sentences are prepared, accounting for the various short-hands that different clinicians might adopt.…”

“…While these techniques have high predictive ability, they can capture relationships between words that reflect biases, e.g., along gender [9] or ethnic identity [21]. While such biases exist within both traditional non-contextual word embeddings [9,73] and contextual word embeddings [36,62,72], biases in pretrained contextual embedding models can additionally encode historical biases in the training corpora, class imbalance in datasets, and data quality differences [53,74]. This is especially concerning with the growth of large pretrained contextual embeddings models, such as variants of the BERT architecture [16,40,50,52,67].…”