Intersectional Bias in Hate Speech and Abusive Language Datasets

Kim, Jae Yeon; Ortíz, Carlos; Nam, Sarah; Santiago, Sarah; Datta, Vivek

doi:10.48550/arxiv.2005.05921

Cited by 12 publications

(15 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even methods developed to prevent harm towards minority groups can exacerbate inequalities. [42] for example found intersectional bias in the Twitter datasets used to train algorithms that detect hate speech and abusive language on social media, noting higher rates of tagged tweets from those posted by African-American men.…”

Section: Social Consequences Of Biasmentioning

confidence: 99%

Intersectional Bias in Causal Language Models

Magee,

Ghahremanlou,

Soldatic

et al. 2021

Preprint

View full text Add to dashboard Cite

To examine whether intersectional bias can be observed in language generation, we examine GPT-2 and GPT-NEO models, ranging in size from 124 million to 2.7 billion parameters. We conduct an experiment combining up to three social categories -gender, religion and disability -into unconditional or zero-shot prompts used to generate sentences that are then analysed for sentiment. Our results confirm earlier tests conducted with auto-regressive causal models, including the GPT family of models. We also illustrate why bias may be resistant to techniques that target single categories (e.g. gender, religion and race), as it can also manifest, in often subtle ways, in texts prompted by concatenated social categories. To address these difficulties, we suggest technical and community-based approaches need to combine to acknowledge and address complex and intersectional language model bias.

show abstract

Section: Social Consequences Of Biasmentioning

confidence: 99%

Intersectional Bias in Causal Language Models

Magee,

Ghahremanlou,

Soldatic

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, the problem of toxic speech online platforms from LMs is not easy to address. Toxicity mitigation techniques have been shown to perpetuate discriminatory biases whereby toxicity detection tools more often falsely flag utterances from historically marginalised groups as toxic (Dixon et al, 2018;Jigsaw, 2021;Kim et al, 2020), and detoxification methods work less well for these same groups (Sap et al, 2019;Welbl et al, 2021).…”

Section: Problemmentioning

confidence: 99%

Ethical and social risks of harm from Language Models

Weidinger¹,

Mellor²,

Rauh³

et al. 2021

Preprint

105

View full text Add to dashboard Cite

This paper aims to help structure the risk landscape associated with large-scale Language Models (LMs). In order to foster advances in responsible innovation, an in-depth understanding of the potential risks posed by these models is needed. A wide range of established and anticipated risks are analysed in detail, drawing on multidisciplinary literature from computer science, linguistics, and social sciences.

show abstract

“…Kim et al [44] talked about bias against race and gender for the African American community in toxicity detection. They used the Founta100k dataset and labeled 𝑝 (𝑁 𝐻𝐵|𝑡𝑤𝑒𝑒𝑡) using the Blodgett LM (c.f.…”

Section: Intersectional Biasmentioning

confidence: 99%

“…We observe that researchers often failed to acknowledge this critical aspect while evaluating their proposed mitigation methods (Section 10.3). Initial work in the direction of intersectional bias has been led by Kim et al [44] who analysed the combined impact of gender and race (Section 9.1). However, the analysis and evaluation of the inter play of various biases on toxicity detection remains an open question.…”

Section: Case Study: Shift In Bias Due To Knowledge-based Generalizat...mentioning

confidence: 99%

Handling Bias in Toxic Speech Detection: A Survey

Garg¹,

Masud²,

Suresh³

et al. 2022

Preprint

View full text Add to dashboard Cite

The massive growth of social media usage has witnessed a tsunami of online toxicity in teams of hate speech, abusive posts, cyberbullying, etc. Detecting online toxicity is challenging due to its inherent subjectivity. Factors such as the context of the speech, geography, socio-political climate, and background of the producers and consumers of the posts play a crucial role in determining if the content can be flagged as toxic. Adoption of automated toxicity detection models in production can lead to a sidelining of the various demographic and psychographic groups they aim to help in the first place. It has piqued researchers' interest in examining unintended biases and their mitigation. Due to the nascent and multi-faceted nature of the work, complete literature is chaotic in its terminologies, techniques, and findings. In this paper, we put together a systematic study to discuss the limitations and challenges of existing methods.We start by developing a taxonomy for categorising various unintended biases and a suite of evaluation metrics proposed to quantify such biases. We take a closer look at each proposed method for evaluating and mitigating bias in toxic speech detection. To examine the limitations of existing methods, we also conduct a case study to introduce the concept of bias shift due to knowledge-based bias mitigation methods. The survey concludes with an overview of the critical challenges, research gaps and future directions.While reducing toxicity on online platforms continues to be an active area of research, a systematic study of various biases and their mitigation strategies will help the research community produce robust and fair models.

show abstract

Intersectional Bias in Hate Speech and Abusive Language Datasets

Cited by 12 publications

References 0 publications

Intersectional Bias in Causal Language Models

Intersectional Bias in Causal Language Models

Ethical and social risks of harm from Language Models

Handling Bias in Toxic Speech Detection: A Survey

Contact Info

Product

Resources

About