Understanding and Interpreting the Impact of User Context in Hate Speech Detection

Mosca, E.; Wich, Maximilian; Groh, Georg

doi:10.18653/v1/2021.socialnlp-1.8

Cited by 22 publications

(9 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although our methodology excludes works that are published in venues other than ACL conferences and workshops, we believe that it gives a good indication of the status of XAI in fairness and bias research in NLP. Mosca et al (2021) use SHAP to demonstrate that adding user features to a hate speech detection model reduces biases that are due to spurious correlations in text, but introduces other biases based on user information. Wich et al (2020) also apply SHAP to two example inputs in order to illustrate the political bias of a hate speech model.…”

Section: Applications Of Xai In Fair Nlpmentioning

confidence: 99%

Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models

Balkir¹,

Kiritchenko²,

Nejadgholi³

et al. 2022

Proceedings of the 2nd Workshop on Trustworthy Natural Language Processing (TrustNLP 2022)

View full text Add to dashboard Cite

Motivations for methods in explainable artificial intelligence (XAI) often include detecting, quantifying and mitigating bias, and contributing to making machine learning models fairer. However, exactly how an XAI method can help in combating biases is often left unspecified. In this paper, we briefly review trends in explainability and fairness in NLP research, identify the current practices in which explainability methods are applied to detect and mitigate bias, and investigate the barriers preventing XAI methods from being used more widely in tackling fairness issues.

show abstract

Section: Applications Of Xai In Fair Nlpmentioning

confidence: 99%

Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models

Balkir¹,

Kiritchenko²,

Nejadgholi³

et al. 2022

Proceedings of the 2nd Workshop on Trustworthy Natural Language Processing (TrustNLP 2022)

View full text Add to dashboard Cite

show abstract

“…(4) As non-grouped random raters representing diversity to understand impact on annotations [26,44] (5) As identity-trained classifier-evaluation where classifiers trained on different pools of raters based on markers like gender, first language, age and education are evaluated as to how they perform in terms of precision and recall [4]. (6) As effects of comment authors' demographic differences and not the annotators' demographics as we study [20,25,27,43], or creating new data sets based on political sub-communities of authors [40]. (7) As focusing on model-robustness-evaluation to detect identity markers like race and gender correctly [36], multi-lingual hate speech classification [5], and model-fairness-evaluation [17,19].…”

Section: Related Workmentioning

confidence: 99%

Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation

Goyal¹,

Kivlichan²,

Rosen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Machine learning models are commonly used to detect toxicity in online conversations. These models are trained on datasets annotated by human raters. We explore how raters' self-described identities impact how they annotate toxicity in online comments. We first define the concept of specialized rater pools: rater pools formed based on raters' self-described identities, rather than at random. We formed three such rater pools for this study-specialized rater pools of raters from the U.S. who identify as African American, LGBTQ, and those who identify as neither. Each of these rater pools annotated the same set of comments, which contains many references to these identity groups. We found that rater identity is a statistically significant factor in how raters will annotate toxicity for identity-related annotations. Using preliminary content analysis, we examined the comments with the most disagreement between rater pools and found nuanced differences in the toxicity annotations. Next, we trained models on the annotations from each of the different rater pools, and compared the scores of these models on comments from several test sets. Finally, we discuss how using raters that self-identify with the subjects of comments can create more inclusive machine learning models, and provide more nuanced ratings than those by random raters. Please be advised that this work contains examples of toxic and offensive content.

show abstract

“…Wang (2018) proposes an adaptation of explainability techniques for computer vision to visualize and understand the CNN-GRU classifier for hate speech (Zhang et al, 2018). Mosca et al (2021) (Bach et al, 2015), and a selfexplanatory model (LSTM with an attention mechanism). SHAP explainer is applied (Wich et al, 2020) to investigate the impact of political bias on hate speech classification.…”

Section: Related Workmentioning

confidence: 99%

Benchmarking Post-Hoc Interpretability Approaches for Transformer-based Misogyny Detection

Attanasio¹,

Nozza²,

Pastor³

et al. 2022

Proceedings of NLP Power! The First Workshop on Efficient Benchmarking in NLP

View full text Add to dashboard Cite

Warning: This paper contains examples of language that some people may find offensive.Transformer-based Natural Language Processing models have become the standard for hate speech detection. However, the unconscious use of these techniques for such a critical task comes with negative consequences. Various works have demonstrated that hate speech classifiers are biased. These findings have prompted efforts to explain classifiers, mainly using attribution methods. In this paper, we provide the first benchmark study of interpretability approaches for hate speech detection. We cover four post-hoc token attribution approaches to explain the predictions of Transformer-based misogyny classifiers in English and Italian. Further, we compare generated attributions to attention analysis. We find that only two algorithms provide faithful explanations aligned with human expectations. Gradient-based methods and attention, however, show inconsistent outputs, making their value for explanations questionable for hate speech detection tasks.

show abstract

Understanding and Interpreting the Impact of User Context in Hate Speech Detection

Cited by 22 publications

References 25 publications

Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models

Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models

Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation

Benchmarking Post-Hoc Interpretability Approaches for Transformer-based Misogyny Detection

Contact Info

Product

Resources

About