"I'm sorry to hear that": finding bias in language models with a holistic descriptor dataset

Abstract

As language models grow in popularity, their biases across all possible markers of demographic identity should be measured and addressed in order to avoid perpetuating existing societal harms. Many datasets for measuring bias currently exist, but they are restricted in their coverage of demographic axes, and are commonly used with preset bias tests that presuppose which types of biases the models exhibit. In this work, we present a new, more inclusive dataset, HOLISTICBIAS, which consists of nearly 600 descriptor terms across 13 different demographic axes. HOLISTICBIAS was assembled in conversation with experts and community members with lived experience through a participatory process. We use these descriptors combinatorially in a set of bias measurement templates to produce over 450,000 unique sentence prompts, and we use these prompts to explore, identify, and reduce novel forms of bias in several generative models. We demonstrate that our dataset is highly efficacious for measuring previously unmeasurable biases in token likelihoods and generations from language models, as well as in an offensiveness classifier. We will invite additions and amendments to the dataset, and we hope it will help serve as a basis for easy-to-use and more standardized methods for evaluating bias in NLP models.

Which Stereotypes Are Moderated and Under-Moderated in Search Engine Autocompletion?

“…In open-ended language generation, prompts are often used to assess to what extent LMs yield undesirable output. Various benchmarks such as BOLD [17], HONEST [49], HolisticBias [61] and RealToxicityPrompts [24] exist for this purpose. Choenni et al [13] prompt language models to assess to what extent they have learnt stereotypes.…”

Measuring Gender Bias in Natural Language Processing: Incorporating Gender-Neutral Linguistic Forms for Non-Binary Gender Identities in Abusive Speech Detection

“…An additional study presents a large GBET dataset called HOLISTICBIAS for measuring bias. This dataset is assembled by using a set of demographic descriptor terms in a set of bias measurement templates and can be used to test bias in language models (Smith et al, 2022).…”

Perturbation Augmentation for Fairer NLP

“…In addition to standard fairness evaluation datasets such as CrowS-Pairs (Nangia et al, 2020) 7 , and template-based fairness 7 Although many of the fairness metrics that are standard in NLP have flaws , we unfortunately have few alternatives. measurements such as the Word Embedding Association Test (WEAT) (Caliskan et al, 2017) and Sentence Encoder Association Test (SEAT), (May et al, 2019)), we also incorporate a new, larger bias measurement dataset, HolisticBias (HB), which was created with a combination of algorithmic and participatory processes to develop the most comprehensive descriptor term list available (Smith et al, 2022). We calculate the per-axis bias by measuring the fraction of pairs of descriptors in the HB dataset for which the distribution of pseudo-loglikelihoods (Nangia et al, 2020) in templated sentences significantly differs.…”