RECAST: Interactive Auditing of Automatic Toxicity Detection Models

Wright, Austin P.; Shaikh, Omar; Park, Haekyu; Epperson, Will; Ahmed, Muhammed; Pinel, Stephane; Yang, Diyi; Chau, Duen Horng

doi:10.1145/3403676.3403691

Cited by 4 publications

(10 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides, they found no evidence that context actually improves the performance of toxicity classifiers. In another work [38] the authors presented an interactive tool for auditing toxicity detection models by visualizing explanations for predictions and providing alternative wordings for detected toxic speech. In particular, they displayed the attention of toxicity detection models on user input, providing suggestions on how to replace sensitive text with less toxic words.…”

Section: Related Workmentioning

confidence: 99%

An Assessment of Deep Learning Models and Word Embeddings for Toxicity Detection within Online Textual Comments

2021

View full text Add to dashboard Cite

Today, increasing numbers of people are interacting online and a lot of textual comments are being produced due to the explosion of online communication. However, a paramount inconvenience within online environments is that comments that are shared within digital platforms can hide hazards, such as fake news, insults, harassment, and, more in general, comments that may hurt someone’s feelings. In this scenario, the detection of this kind of toxicity has an important role to moderate online communication. Deep learning technologies have recently delivered impressive performance within Natural Language Processing applications encompassing Sentiment Analysis and emotion detection across numerous datasets. Such models do not need any pre-defined hand-picked features, but they learn sophisticated features from the input datasets by themselves. In such a domain, word embeddings have been widely used as a way of representing words in Sentiment Analysis tasks, proving to be very effective. Therefore, in this paper, we investigated the use of deep learning and word embeddings to detect six different types of toxicity within online comments. In doing so, the most suitable deep learning layers and state-of-the-art word embeddings for identifying toxicity are evaluated. The results suggest that Long-Short Term Memory layers in combination with mimicked word embeddings are a good choice for this task.

show abstract

Section: Related Workmentioning

confidence: 99%

An Assessment of Deep Learning Models and Word Embeddings for Toxicity Detection within Online Textual Comments

2021

View full text Add to dashboard Cite

show abstract

“…These can be used to show how changing the input features affects the distribution of predictions and counterfactuals, especially for systems that support users in verifying and refining the user hypotheses. Conversely, when the focus is on individual instances, it is important to highlight both the differences between the input and the counterfactuals (Figure 10a), and the difference in terms of predictions [WSP*21, SGPR18], for example, by using tables [WPB*19, CMQ21], enhanced representation of the input [CBN*20] (e.g. heatmaps for images) or colour coding.…”

Section: Papers Categorizationmentioning

confidence: 99%

“…Combined with interactions, they allow users to investigate and analyse the models. Examples of added information are the attribution scores’ magnitude, which is encoded using colours, size [JCM20], opacity [WSP*21] or just its value, or bounding boxes [HSL*21, JKV*22, CBN*20], which highlight the most important region over images. While sorting [CHS20, MXC*20, KCK*19, Vig19, PCN*19] and filtering [WONM18, DWSZ20, JKV*22, JTH*21] by attribution scores capabilities are quite common to ease the data exploration and reduce the visual clutters, some works provide additional tools for a deeper understanding.…”

Section: Papers Categorizationmentioning

confidence: 99%

“…Counterfactual examples [WMR17, WPB*19] correspond to samples different but close to the current input and associated with a different prediction. VA systems use them to support users in verifying and refining the hypotheses about the decision process on selected data instances [CMQ21, SGPR18], in applying edits to input to obtain a different decision [WPB*19, WSP*21, LLL*19, WM20] or in exploring alternative scenarios on reinforcement learning agents [MSHB22]. These explanations can be employed when the DL model is deployed and static, and the user wants to understand the change one has to make to the input sample to obtain a modified output.…”

Section: Papers Categorizationmentioning

confidence: 99%

“…The user can interact and customize counterfactuals by changing the value of the input features, setting constraints that counterfactuals should satisfy, or setting their number. (b) Recast [WSP*21] exploits counterfactuals to help user to lower the toxicity score of their text. When the user hovers over a word or a part of the sentence, the systems show counterfactuals in terms of words that the user should replace to obtain a different score.…”

Section: Papers Categorizationmentioning

confidence: 99%

See 2 more Smart Citations

State of the Art of Visual Analytics for eXplainable Deep Learning

Rosa

Blasilli

Bourqui

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

The use and creation of machine-learning-based solutions to solve problems or reduce their computational costs are becoming increasingly widespread in many domains. Deep Learning plays a large part in this growth. However, it has drawbacks such as a lack of explainability and behaving as a black-box model. During the last few years, Visual Analytics has provided several proposals to cope with these drawbacks, supporting the emerging eXplainable Deep Learning field. This survey aims to (i) systematically report the contributions of Visual Analytics for eXplainable Deep Learning; (ii) spot gaps and challenges; (iii) serve as an anthology of visual analytical solutions ready to be exploited and put into operation by the Deep Learning community (architects, trainers and end users) and (iv) prove the degree of maturity, ease of integration and results for specific domains. The survey concludes by identifying future research challenges and bridging activities that are helpful to strengthen the role of Visual Analytics as effective support for eXplainable Deep Learning and to foster the adoption of Visual Analytics solutions in the eXplainable Deep Learning community. An interactive explorable version of this survey is available online at https:// awarediag-sapienza.github.io/ VA4XDL.

show abstract

Linguistically Differentiating Acts and Recalls of Racial Microaggressions on Social Media

Gunturi,

Kumar,

Ding

et al. 2024

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

In this work, we examine the linguistic signature of online racial microaggressions (acts) and how it differs from that of personal narratives recalling experiences of such aggressions (recalls) by Black social media users. We manually curate and annotate a corpus of acts and recalls fromin-the-wild social media discussions, and verify labels with Black workshop participants. We leverage Natural Language Processing (NLP) and qualitative analysis on this data to classify (RQ1), interpret (RQ2), and characterize (RQ3) the language underlying acts and recalls of racial microaggressions in the context of racism in the U.S. Our findings show that neural language models (LMs) can classify acts and recalls with high accuracy (RQ1) with contextual words revealing themes that associate Blacks with objects that reify negative stereotypes (RQ2). Furthermore, overlapping linguistic signatures between acts and recalls serve functionally different purposes (RQ3), providing broader implications to the current challenges in content moderation systems on social media.

show abstract

RECAST: Interactive Auditing of Automatic Toxicity Detection Models

Cited by 4 publications

References 2 publications

An Assessment of Deep Learning Models and Word Embeddings for Toxicity Detection within Online Textual Comments

An Assessment of Deep Learning Models and Word Embeddings for Toxicity Detection within Online Textual Comments

State of the Art of Visual Analytics for eXplainable Deep Learning

Linguistically Differentiating Acts and Recalls of Racial Microaggressions on Social Media

Contact Info

Product

Resources

About