Six Attributes of Unhealthy Conversations

Price, Ilan; Gifford-Moore, Jordan; Flemming, Jory; Musker, Saul; Roichman, Maayan; Sylvain, Guillaume; Thain, Nithum; Dixon, Lucas; Sorensen, Jeffrey

doi:10.18653/v1/2020.alw-1.15

Cited by 30 publications

(24 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent work coming out of a collaboration between Google and researchers from the universities of Oxford and South Carolina, these operative distinctions are becoming more fine-grained, seeking to tackle “subtle forms of toxicity” (Price et al, 2020), which means identifying comments that are “(1) hostile; (2) antagonistic, insulting, provocative or trolling; (3) dismissive; (4) condescending or patronizing; (5) sarcastic; and/or (6) an unfair generalisation” (Price et al, 2020: 1). Whether this work ends up in a production system or not, it signals the deep entanglement between technical design and potentially far-reaching moments of normative prescription in machine moderation.…”

Section: The Fabrics Of Powermentioning

confidence: 99%

The fabrics of machine moderation: Studying the technical, normative, and organizational structure of Perspective API

Rieder

Skop

2021

Big Data & Society

View full text Add to dashboard Cite

Over recent years, the stakes and complexity of online content moderation have been steadily raised, swelling from concerns about personal conflict in smaller communities to worries about effects on public life and democracy. Because of the massive growth in online expressions, automated tools based on machine learning are increasingly used to moderate speech. While ‘design-based governance’ through complex algorithmic techniques has come under intense scrutiny, critical research covering algorithmic content moderation is still rare. To add to our understanding of concrete instances of machine moderation, this article examines Perspective API, a system for the automated detection of ‘toxicity’ developed and run by the Google unit Jigsaw that can be used by websites to help moderate their forums and comment sections. The article proceeds in four steps. First, we present our methodological strategy and the empirical materials we were able to draw on, including interviews, documentation, and GitHub repositories. We then summarize our findings along five axes to identify the various threads Perspective API brings together to deliver a working product. The third section discusses two conflicting organizational logics within the project, paying attention to both critique and what can be learned from the specific case at hand. We conclude by arguing that the opposition between ‘human’ and ‘machine’ in speech moderation obscures the many ways these two come together in concrete systems, and suggest that the way forward requires proactive engagement with the design of technologies as well as the institutions they are embedded in.

show abstract

Section: The Fabrics Of Powermentioning

confidence: 99%

The fabrics of machine moderation: Studying the technical, normative, and organizational structure of Perspective API

Rieder

Skop

2021

Big Data & Society

View full text Add to dashboard Cite

show abstract

“…NLP-aided approaches to detect abusive behavior online is an active research area (Schmidt and Wiegand, 2017;Mishra et al, 2019;Corazza et al, 2020). Researchers have developed typologies of online abuse (Waseem et al, 2017), constructed datasets annotated with different types of abusive language (Warner and Hirschberg, 2012;Price et al, 2020;Vidgen et al, 2021), and built NLP models to detect them efficiently Mozafari et al, 2019). Researchers have also expanded the focus to more subtle forms of abuse such as condescension and microaggressions (Breitfeller et al, 2019;.…”

Section: Detecting Online Abusementioning

confidence: 99%

Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations

Davani¹,

Díaz²,

Prabhakaran³

2021

Preprint

View full text Add to dashboard Cite

Majority voting and averaging are common approaches employed to resolve annotator disagreements and derive single ground truth labels from multiple annotations. However, annotators may systematically disagree with one another, often reflecting their individual biases and values, especially in the case of subjective tasks such as detecting affect, aggression, and hate speech. Annotator disagreements may capture important nuances in such tasks that are often ignored while aggregating annotations to a single ground truth. In order to address this, we investigate the efficacy of multi-annotator models. In particular, our multi-task based approach treats predicting each annotators' judgements as separate subtasks, while sharing a common learned representation of the task. We show that this approach yields same or better performance than aggregating labels in the data prior to training across seven different binary classification tasks. Our approach also provides a way to estimate uncertainty in predictions, which we demonstrate better correlate with annotation disagreements than traditional methods. Being able to model uncertainty is especially useful in deployment scenarios where knowing when not to make a prediction is important.

show abstract

“…In another line of work, theoretical foundations are being established, in the form of taxonomies (Banko et al, 2020), definitions (Wiegand et al, 2021;Waseem et al, 2017) and theory (Price et al, 2020;Laaksonen et al, 2020). We are adding to this with definitions based on fieldwork and grounded research, inspired by anthropological and ethnographic work that investigates the societal impact of online hate and extreme speech (Boromisza-Habashi, 2013;Donovan and danah boyd, 2021;Haynes, 2019;Udupa and Pohjonen, 2019;Hervik, 2019).…”

Section: Related Workmentioning

confidence: 99%

Listening to Affected Communities to Define Extreme Speech: Dataset and Experiments

Maronikolakis¹,

Wisiorek²,

Nann³

et al. 2022

Preprint

View full text Add to dashboard Cite

Building on current work on multilingual hate speech (e.g., Ousidhoum et al. (2019)) and hate speech reduction (e.g., Sap et al. ( 2020)), we present XTREMESPEECH, 1 a new hate speech dataset containing 20,297 social media passages from Brazil, Germany, India and Kenya. The key novelty is that we directly involve the affected communities in collecting and annotating the data -as opposed to giving companies and governments control over defining and combatting hate speech. This inclusive approach results in datasets more representative of actually occurring online speech and is likely to facilitate the removal of the social media content that marginalized communities view as causing the most harm. Based on XTREMESPEECH, we establish novel tasks with accompanying baselines, provide evidence that cross-country training is generally not feasible due to cultural differences between countries and perform an interpretability analysis of BERT's predictions.

show abstract

Six Attributes of Unhealthy Conversations

Cited by 30 publications

References 25 publications

The fabrics of machine moderation: Studying the technical, normative, and organizational structure of Perspective API

The fabrics of machine moderation: Studying the technical, normative, and organizational structure of Perspective API

Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations

Listening to Affected Communities to Define Extreme Speech: Dataset and Experiments

Contact Info

Product

Resources

About