Fighting Adversarial Attacks on Online Abusive Language Moderation

Rodríguez, Néstor; Rojas–Galeano, Sergio

doi:10.1007/978-3-030-00350-0_40

Cited by 3 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Harmful [91,156,245] and toxic languages [113,166,224] are not defined precisely in literature, except for toxicity in video games. However, toxic speech is described in a crowdsourcing task to collect a dataset of toxic comments [292] 7 and insists on the type of language used, which motivated our characterization choice.…”

Section: Methodsmentioning

confidence: 99%

“…Park et al [193] use the False Positive and False Negative Equality Differences to quantify gender biases. Some publications assess the time taken to train the models or the time to detect the OCL [165,215,224,300]. Some papers further study the performance of the models by investigating in more detail the types of sentences usually missclassified.…”

Section: Evaluation Metricmentioning

confidence: 99%

“…The design of rules is often done in two steps. Dictionaries expressing OCL are prepared [11,224,296] (e.g., subjectivity lexicon, hate lexicon, and list of haterepresentative grammatical relations [83], lists of profane words augmented with genomicsinspired techniques [225]). Then, lists of rules are defined to attribute a score to samples based on their use of the dictionary vocabulary, often using pattern or word matching.…”

Section: A6 Summary Of Common Classification Algorithmsmentioning

confidence: 99%

See 2 more Smart Citations

Automatic Identification of Harmful, Aggressive, Abusive, and Offensive Language on the Web: A Survey of Technical Biases Informed by Psychology Literature

Balayn

Yang

Szlávik

et al. 2021

Trans. Soc. Comput.

View full text Add to dashboard Cite

The automatic detection of conflictual languages (harmful, aggressive, abusive, and offensive languages) is essential to provide a healthy conversation environment on the Web. To design and develop detection systems that are capable of achieving satisfactory performance, a thorough understanding of the nature and properties of the targeted type of conflictual language is of great importance. The scientific communities investigating human psychology and social behavior have studied these languages in details, but their insights have only partially reached the computer science community. In this survey, we aim both at systematically characterizing the conceptual properties of online conflictual languages, and at investigating the extent to which they are reflected in state-of-the-art automatic detection systems. Through an analysis of psychology literature, we provide a reconciled taxonomy that denotes the ensemble of conflictual languages typically studied in computer science. We then characterize the conceptual mismatches that can be observed in the main semantic and contextual properties of these languages and their treatment in computer science works; and systematically uncover resulting technical biases in the design of machine learning classification models and the dataset created for their training. Finally, we discuss diverse research opportunities for the computer science community and reflect on broader technical and structural issues.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Evaluation Metricmentioning

confidence: 99%

Section: A6 Summary Of Common Classification Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Identification of Harmful, Aggressive, Abusive, and Offensive Language on the Web: A Survey of Technical Biases Informed by Psychology Literature

Balayn

Yang

Szlávik

et al. 2021

Trans. Soc. Comput.

View full text Add to dashboard Cite

show abstract

“…These efforts, however, are not scalable enough to curtail the rapid growth of toxic content on online platforms ( Davidson et al, 2017 ). There is also the psychological distress associated with exposing human moderators to firsthand accounts of toxicity ( Rodriguez & Rojas-Galeano, 2018 ). These challenges call for developing effective automatic or semiautomatic solutions to detect toxicity from a large stream of content on online platforms.…”

Section: Introductionmentioning

confidence: 99%

“…Users of social media platforms have various reasons for spreading harmful content, like personal or social gain ( Squicciarini, Dupont & Chen, 2014 ). Studies show that publishing toxic content ( i.e., toxic behavior) is contagious ( Tsikerdekis & Zeadally, 2014 ; Rodriguez & Rojas-Galeano, 2018 ); the malicious behavior of users can influence non-malicious users and leads them to misbehave, which affects the overall well-being of online communities. As an example of toxic behavior ( Alfonso & Morris, 2013 ), one Reddit user named Violentacrez created several communities on controversial topics such as gore, and his followers mimicked this behavior by creating communities with highly offensive content as well.…”

Section: Introductionmentioning

confidence: 99%

Investigating toxicity changes of cross-community redditors from 2 billion posts and comments

Almerekhi

Kwak

Jansen

2022

PeerJ Computer Science

View full text Add to dashboard Cite

This research investigates changes in online behavior of users who publish in multiple communities on Reddit by measuring their toxicity at two levels. With the aid of crowdsourcing, we built a labeled dataset of 10,083 Reddit comments, then used the dataset to train and fine-tune a Bidirectional Encoder Representations from Transformers (BERT) neural network model. The model predicted the toxicity levels of 87,376,912 posts from 577,835 users and 2,205,581,786 comments from 890,913 users on Reddit over 16 years, from 2005 to 2020. This study utilized the toxicity levels of user content to identify toxicity changes by the user within the same community, across multiple communities, and over time. As for the toxicity detection performance, the BERT model achieved a 91.27% classification accuracy and an area under the receiver operating characteristic curve (AUC) score of 0.963 and outperformed several baseline machine learning and neural network models. The user behavior toxicity analysis showed that 16.11% of users publish toxic posts, and 13.28% of users publish toxic comments. However, results showed that 30.68% of users publishing posts and 81.67% of users publishing comments exhibit changes in their toxicity across different communities, indicating that users adapt their behavior to the communities’ norms. Furthermore, time series analysis with the Granger causality test of the volume of links and toxicity in user content showed that toxic comments are Granger caused by links in comments.

show abstract