SemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter

Basile, Valerio; Bosco, Cristina; Fersini, Elisabetta; Nozza, Debora; Patti, Viviana; Pardo, Francisco M. Rangel; Rosso, Paolo; Sanguinetti, Manuela

doi:10.18653/v1/s19-2007

Cited by 525 publications

(541 citation statements)

References 12 publications

Supporting

Mentioning

532

Contrasting

Unclassified

Order By: Relevance

“…Note that for the Subtask B, evaluation was based on two criteria (each dimension evaluated independently or jointly), however the final ranking was based solely on the second criteria (Exact Match Ratio on the three labels). More details about the evaluation system can be found in the task description paper (Basile et al, 2019).…”

Section: Resultsmentioning

confidence: 99%

“…In this paper, we describe our system for Task 5 of SemEval 2019 (Basile et al, 2019), namely Multilingual detection of hate speech against immigrants and women in Twitter (HatEval). In this task, participants were asked to automatically classify English and Spanish tweets as hateful or not for Subtask A, and to predict if these tweets are aggressive or not, then identify whether the target is generic or individual for Subtask B.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Saagie at Semeval-2019 Task 5: From Universal Text Embeddings and Classical Features to Domain-specific Text Classification

Benballa¹,

Collet²,

Picot-Clémente³

2019

Proceedings of the 13th International Workshop on Semantic Evaluation

View full text Add to dashboard Cite

This paper describes our contribution to Se-mEval 2019 Task 5: Hateval. We propose to investigate how domain-specific text classification task can benefit from pretrained state of the art language models and how they can be combined with classical handcrafted features. For this purpose, we propose an approach based on a feature-level Meta-Embedding to let the model choose which features to keep and how to use them.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Saagie at Semeval-2019 Task 5: From Universal Text Embeddings and Classical Features to Domain-specific Text Classification

Benballa¹,

Collet²,

Picot-Clémente³

2019

Proceedings of the 13th International Workshop on Semantic Evaluation

View full text Add to dashboard Cite

show abstract

“…This shared task [5] aims specifically at detecting hate speech against immigrants and women on Twitter for Spanish and English. HateEval is offered in two subtasks.…”

Section: Hateeval: Semeval 2019 Taskmentioning

confidence: 99%

“…Contextual language models, such as BERT [13] and ELMo [40], have shown promising results at different evaluation forums such as Offenseval [69] or HateEval [5]. Many HASOC Teams, such as BRUMS [42], Raligraph [25], 3Idiots [27], LSV-UDS [10] and KMI-Panlingua [44], have done classification using BERT.…”

Section: Transfer Learning Modelmentioning

confidence: 99%

Tracking Hate in Social Media: Evaluation, Challenges and Approaches

et al. 2020

View full text Add to dashboard Cite

This paper presents online hate speech as a societal and computational challenge. Offensive content detection in social media is considered as a multilingual, multi-level, multi-class classification problem for three Indo-European languages. This research problem is offered to the community through the HASOC shared task. HASOC intends to stimulate research and development in hate speech recognition across different languages. Three datasets (in English, German, and Hindi) were developed from Twitter and Facebook, and made available. This paper describes the creation of the multilingual datasets and the annotation method. We will present the numerous approaches based on traditional classifiers, deep neural models, and transfer learning models, along with features used for the classification. Results show that the best classifier for the binary classification might not perform best in the multi-class classification, and the performance of the same classifier varies across the languages. Overall, transfer learning models such as BERT, and deep neural models based on LSTMs and CNNs perform similar but better than traditional classifiers such as SVM. We will conclude the discussion with a list of issues that needs to be addressed for future datasets.

show abstract

“…Prior work on detecting harmful behavior like hate speech has focused on individual documents such as blog posts or comments (Spertus, 1997;Magu et al, 2017;Pavlopoulos et al, 2017;Davidson et al, 2017;de la Vega and Ng, 2018;Basile et al, 2019;Zampieri et al, 2019). Recently, there have been some efforts to incorporate userlevel information.…”

Section: Related Workmentioning

confidence: 99%

Learning Invariant Representations of Social Media Users

Andrews

Bishop²

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

The evolution of social media users' behavior over time complicates user-level comparison tasks such as verification, classification, clustering, and ranking. As a result, naïve approaches may fail to generalize to new users or even to future observations of previously known users. In this paper, we propose a novel procedure to learn a mapping from short episodes of user activity on social media to a vector space in which the distance between points captures the similarity of the corresponding users' invariant features. We fit the model by optimizing a surrogate metric learning objective over a large corpus of unlabeled social media content. Once learned, the mapping may be applied to users not seen at training time and enables efficient comparisons of users in the resulting vector space. We present a comprehensive evaluation to validate the benefits of the proposed approach using data from Reddit, Twitter, and Wikipedia.

show abstract

SemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter

Cited by 525 publications

References 12 publications

Saagie at Semeval-2019 Task 5: From Universal Text Embeddings and Classical Features to Domain-specific Text Classification

Saagie at Semeval-2019 Task 5: From Universal Text Embeddings and Classical Features to Domain-specific Text Classification

Tracking Hate in Social Media: Evaluation, Challenges and Approaches

Learning Invariant Representations of Social Media Users

Contact Info

Product

Resources

About