ParaDetox: Detoxification with Parallel Data

Logacheva, Varvara; Dementieva, Daryna; Ustyantsev, Sergey; Moskovskiy, Daniil; Dale, David C.; Кротова, И. Н.; Semenov, Nikita; Panchenko, Alexander

doi:10.18653/v1/2022.acl-long.469

Cited by 15 publications

(18 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, our results suggest that solely cleaning the generated continuations leads to a remarkable PPL deterioration. As Logacheva et al (2022) demonstrates that their method produces fluent cleaned text, this deterioration could be attributed to a loss of context relevance, rather than fluency issues. Further, this approach does not significantly reduce toxicity.…”

Section: Continuationmentioning

confidence: 99%

See 1 more Smart Citation

Detoxifying Language Models with a Toxic Corpus

Park¹,

Rudzicz²

2022

Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion

View full text Add to dashboard Cite

Existing studies have investigated the tendency of autoregressive language models to generate contexts that exhibit undesired biases and toxicity. Various debiasing approaches have been proposed, which are primarily categorized into data-based and decoding-based. In our study, we investigate the ensemble of the two debiasing paradigms, proposing to use toxic corpus as an additional resource to reduce the toxicity. Our result shows that toxic corpus can indeed help to reduce the toxicity of the language generation process substantially, complementing the existing debiasing methods.

show abstract

Section: Continuationmentioning

confidence: 99%

“…Nevertheless, one can obtain nontoxic texts by generating them and then detoxifying them. Thus, we provide an additional experiment comparing one text detoxification method, bartdetox-base (Logacheva et al, 2022), with our LM detoxification one.…”

Section: Continuationmentioning

confidence: 99%

Detoxifying Language Models with a Toxic Corpus

Park¹,

Rudzicz²

2022

Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion

View full text Add to dashboard Cite

show abstract

“…Some researchers found that only a small proportion of the whole response (e.g., one or two words) needs to be fixed. Thus, An edition module takes effect after the generation to fix some problems in some works [75,78]. Similarly, text style transfer or rephrasing from toxicity to non-toxicity can also be plugged in this stage [26,66].…”

Section: Towards Pipeline-based Systemmentioning

confidence: 99%

Recent Advances towards Safe, Responsible, and Moral Dialogue Systems: A Survey

Deng¹,

Sun²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

With the development of artificial intelligence, dialogue systems have been endowed with amazing chitchat capabilities, and there is widespread interest and discussion about whether the generated contents are socially beneficial. In this paper, we present a new perspective of research scope towards building a safe, responsible, and modal dialogue system, including 1) abusive and toxic contents, 2) unfairness and discrimination, 3) ethics and morality issues, and 4) risk of misleading and privacy information. Besides, we review the mainstream methods for evaluating the safety of large models from the perspectives of exposure and detection of safety issues. The recent advances in methodologies for the safety improvement of both end-to-end dialogue systems and pipeline-based models are further introduced. Finally, we discussed six existing challenges towards responsible AI: explainable safety monitoring, continuous learning of safety issues, robustness against malicious attacks, multimodal information processing, unified research framework, and multidisciplinary theory integration. We hope this survey will inspire further research toward safer dialogue systems. 1

show abstract

“…In previous work on detoxification methods, such kind of datasets were used to develop and test unsupervised text style transfer approaches (Wu et al, 2019;Tran et al, 2020;Hallinan et al, 2022). However, lately a parallel dataset ParaDetox for training supervised text detoxification models for English was released (Logacheva et al, 2022b) similar to previous parallel TST datasets for formality (Rao and Tetreault, 2018;Briakou et al, 2021). Pairs of toxic-neutral sentences were collected with a pipeline based on three crowdsourcing tasks.…”

Section: Related Workmentioning

confidence: 99%

“…Firstly, several unsupervised methods based on masked language modelling (Tran et al, 2020; and disentangled representations for style and content (John et al, 2019;dos Santos et al, 2018) were explored. More recently, Logacheva et al (2022b) showed the superiority of supervised seq2seq models for detoxification trained on a parallel corpus of crowdsourced toxic ↔ neutral sentence pairs. Afterwards, there were experiments in multilingual detoxification.…”

Section: Introductionmentioning

confidence: 99%

Methods for Detoxification of Texts for the Russian Language

Dementieva¹,

Moskovskiy²,

Logacheva³

et al. 2021

Computational Linguistics and Intellectual Technologies

View full text Add to dashboard Cite

We introduce the first study of automatic detoxification of Russian texts to combat offensive language. Such a kind of textual style transfer can be used, for instance, for processing toxic content in social media. While much work has been done for the English language in this field, it has never been solved for the Russian language yet. We test two types of models -unsupervised approach based on BERT architecture that performs local corrections and supervised approach based on pretrained language GPT-2 model -and compare them with several baselines. In addition, we describe evaluation setup providing training datasets and metrics for automatic evaluation. The results show that the tested approaches can be successfully used for detoxification, although there is room for improvement.

show abstract

ParaDetox: Detoxification with Parallel Data

Cited by 15 publications

References 0 publications

Detoxifying Language Models with a Toxic Corpus

Detoxifying Language Models with a Toxic Corpus

Recent Advances towards Safe, Responsible, and Moral Dialogue Systems: A Survey

Methods for Detoxification of Texts for the Russian Language

Contact Info

Product

Resources

About