Annotating Online Misogyny

Zeinert, Philine; Inie, Nanna; Derczynski, Leon

doi:10.18653/v1/2021.acl-long.247

Cited by 19 publications

(29 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Yet other works have grouped annotators into pools by forming clusters of annotators who rate similarly [2,3,6] or by using community-detection algorithms on a graph of annotators [39]. Other have incorporated a diversity of annotators in their rater pools but not explicitly grouped annotators according to identity and drawn conclusions based on identity groups specifically [44].…”

Section: Related Workmentioning

confidence: 99%

Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation

Goyal¹,

Kivlichan²,

Rosen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Machine learning models are commonly used to detect toxicity in online conversations. These models are trained on datasets annotated by human raters. We explore how raters' self-described identities impact how they annotate toxicity in online comments. We first define the concept of specialized rater pools: rater pools formed based on raters' self-described identities, rather than at random. We formed three such rater pools for this study-specialized rater pools of raters from the U.S. who identify as African American, LGBTQ, and those who identify as neither. Each of these rater pools annotated the same set of comments, which contains many references to these identity groups. We found that rater identity is a statistically significant factor in how raters will annotate toxicity for identity-related annotations. Using preliminary content analysis, we examined the comments with the most disagreement between rater pools and found nuanced differences in the toxicity annotations. Next, we trained models on the annotations from each of the different rater pools, and compared the scores of these models on comments from several test sets. Finally, we discuss how using raters that self-identify with the subjects of comments can create more inclusive machine learning models, and provide more nuanced ratings than those by random raters. Please be advised that this work contains examples of toxic and offensive content.

show abstract

Section: Related Workmentioning

confidence: 99%

Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation

Goyal¹,

Kivlichan²,

Rosen³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Another paper published at ACL 2021 studying online misogyny (Zeinert et al, 2021) prompted: large amounts of online abuse and doxxing directed at the authors by name; frivolous freedom of information requests explicitly for the purpose of wasting time; complaints made to the authors' external funding organisations; public attacks from politicians against the authors and their institution; and pejorative opinion articles in the national press against the research. Researchers publishing research about hate speech, misinformation, bias or other forms of textual harms should be aware of and prepared for these kinds of interactions, even though this is far from the norm for most areas of academic research.…”

Section: Preparing For Releasing Textual Harm Researchmentioning

confidence: 99%

Handling and Presenting Harmful Text

Derczynski¹,

Kirk²,

Birhane³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Textual data can pose a risk of serious harm. These harms can be categorised along three axes: (1) the harm type (e.g. misinformation, hate speech or racial stereotypes) (2) whether it is elicited as a feature of the research design from directly studying harmful content (e.g. training a hate speech classifier or auditing unfiltered large-scale datasets) versus spuriously invoked from working on unrelated problems (e.g. language generation or part of speech tagging) but with datasets that nonetheless contain harmful content, and (3) who it affects, from the humans (mis)represented in the data to those handling or labelling the data to readers and reviewers of publications produced from the data. It is an unsolved problem in NLP as to how textual harms should be handled, presented, and discussed; but, stopping work on content which poses a risk of harm is untenable. Accordingly, we provide practical advice and introduce HARMCHECK, a resource for reflecting on research into textual harms. We hope our work encourages ethical, responsible, and respectful research in the NLP community.

show abstract

“…Annotator errors can be found using noise identification techniques (e.g., Hovy et al, 2013;Zhang et al, 2017;Paun et al, 2018;Northcutt et al, 2021), corrected by expert annotators (Vidgen and Derczynski, 2020;Vidgen et al, 2021a) or their impact mitigated by label aggregation. Guidelines which are unclear or incomplete need to be improved by dataset creators, which may require iterative approaches to annotation (Founta et al, 2018;Zeinert et al, 2021). Therefore, quality assurance under the prescriptive paradigm is a laborious but structured process, with inter-annotator agreement as a useful, albeit noisy, measure of dataset quality.…”

Section: Key Benefitsmentioning

confidence: 99%

Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks

Röttger¹,

Vidgen²,

Hovy³

et al. 2021

Preprint

View full text Add to dashboard Cite

Labelled data is the foundation of most natural language processing tasks. However, labelling data is difficult and there often are diverse valid beliefs about what the correct data labels should be. So far, dataset creators have acknowledged annotator subjectivity, but not actively managed it in the annotation process. This has led to partly-subjective datasets that fail to serve a clear downstream use. To address this issue, we propose two contrasting paradigms for data annotation. The descriptive paradigm encourages annotator subjectivity, whereas the prescriptive paradigm discourages it. Descriptive annotation allows for the surveying and modelling of different beliefs, whereas prescriptive annotation enables the training of models that consistently apply one belief. We discuss benefits and challenges in implementing both paradigms, and argue that dataset creators should explicitly aim for one or the other to facilitate the intended use of their dataset. Lastly, we design an annotation experiment to illustrate the contrast between the two paradigms.

show abstract

Annotating Online Misogyny

Cited by 19 publications

References 54 publications

Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation

Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation

Handling and Presenting Harmful Text

Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks

Contact Info

Product

Resources

About