Exploring Cross-lingual Text Detoxification with Large Multilingual Language Models.

Moskovskiy, Daniil; Dementieva, Daryna; Panchenko, Alexander

doi:10.18653/v1/2022.acl-srw.26

Cited by 5 publications

(6 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In most TST papers (e.g. [28,13,22]) the methods are evaluated with the combination of the measures that score the basic TST properties: style accuracy, content preservation, and fluency. Our work is dedicated to formality transfer with an additional task to preserve particular slots.…”

Section: Discussionmentioning

confidence: 99%

Don’t Lose the Message While Paraphrasing: A Study on Content Preserving Style Transfer

Nikolay

Dale²,

Gusev³

et al. 2023

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Text style transfer techniques are gaining popularity in natural language processing allowing paraphrasing text in the required form: from toxic to neural, from formal to informal, from old to the modern English language, etc. Solving the task is not sufficient to generate some neural/informal/modern text, but it is important to preserve the original content unchanged. This requirement becomes even more critical in some applications such as style transfer of goal-oriented dialogues where the factual information shall be kept to preserve the original message, e.g. ordering a certain type of pizza to a certain address at a certain time. The aspect of content preservation is critical for real-world applications of style transfer studies, but it has received little attention. To bridge this gap we perform a comparison of various style transfer models on the example of the formality transfer domain. To perform a study of the content preservation abilities of various style transfer methods we create a parallel dataset of formal vs. informal task-oriented dialogues. The key difference between our dataset and the existing ones like GYAFC [17] is the presence of goal-oriented dialogues with predefined semantic slots essential to be kept during paraphrasing, e.g. named entities. This additional annotation allowed us to conduct a precise comparative study of several state-of-the-art techniques for style transfer. Another result of our study is a modification of the unsupervised method LEWIS [19] which yields a substantial improvement over the original method and all evaluated baselines on the proposed task.

show abstract

Section: Discussionmentioning

confidence: 99%

Don’t Lose the Message While Paraphrasing: A Study on Content Preserving Style Transfer

Nikolay

Dale²,

Gusev³

et al. 2023

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…For text detoxification, only in (Moskovskiy et al, 2022) cross-lingual setup was explored through the translation of inputs and outputs of a monolingual system. It has been shown that detoxification trained for English using a multilingual Transformer is not working for Russian (and vice versa).…”

Section: Cross-lingual Knowledge Transfermentioning

confidence: 99%

“…Afterwards, there were experiments in multilingual detoxification. However, crosslingual transfer between languages with multilingual seq2seq models was shown to be a challenging task (Moskovskiy et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

“…

Text detoxification is the task of transferring the style of text from toxic to neutral. While there are approaches yielding promising results in monolingual setup, e.g., Hallinan et al, 2022), cross-lingual transfer for this task remains a challenging open problem (Moskovskiy et al, 2022). In this work, we present a large-scale study of strategies for cross-lingual text detoxification -given a parallel detoxification corpus for one language; the goal is to transfer detoxification ability to another language for which we do not have such a corpus.

…”

mentioning

confidence: 99%

See 1 more Smart Citation

Methods for Detoxification of Texts for the Russian Language

Dementieva¹,

Moskovskiy²,

Logacheva³

et al. 2021

Computational Linguistics and Intellectual Technologies

View full text Add to dashboard Cite

We introduce the first study of automatic detoxification of Russian texts to combat offensive language. Such a kind of textual style transfer can be used, for instance, for processing toxic content in social media. While much work has been done for the English language in this field, it has never been solved for the Russian language yet. We test two types of models -unsupervised approach based on BERT architecture that performs local corrections and supervised approach based on pretrained language GPT-2 model -and compare them with several baselines. In addition, we describe evaluation setup providing training datasets and metrics for automatic evaluation. The results show that the tested approaches can be successfully used for detoxification, although there is room for improvement.

show abstract

“…For languages with unique linguistic characteristics and cultural contexts [8], such as Vietnamese, acquiring suitable data becomes a formidable challenge. Being a tonal language rich in diacritics and idiomatic expressions, it makes the collection and curation of data in Vietnamese even more critical due to its distinctiveness from languages like English [9]. This paper embarks on an exploration of the intricacies involved in collecting and curating data for fine-tuning Vietnamese language models.…”

Section: Introductionmentioning

confidence: 99%

A study on factors affecting the decision of choosing university of high school pupils in Quang Ngai province

Nguyen¹

2021

VMOST

View full text Add to dashboard Cite

In fact, choosing a career and a university of high school pupils is very important. This study aims at identifying and measuring the factors affecting the decision of choosing a university of high school pupils in Quang Ngai province. The data of this study were collected from 340 answer sheets of grade 12 pupils of 5 high schools in Quang Ngai province in the 2019-2020 school year. By using the quantitative analysis methods, it has been shown that 5 factors are affecting the decision of choosing a university of pupils in Quang Ngai province, in the order of high importance to low including (1) University reputation, (2) Communication, (3) Learning conditions, (4) Factor belongs to the pupils, (5) Individuals.

show abstract

Exploring Cross-lingual Text Detoxification with Large Multilingual Language Models.

Cited by 5 publications

References 25 publications

Don’t Lose the Message While Paraphrasing: A Study on Content Preserving Style Transfer

Don’t Lose the Message While Paraphrasing: A Study on Content Preserving Style Transfer

Methods for Detoxification of Texts for the Russian Language

A study on factors affecting the decision of choosing university of high school pupils in Quang Ngai province

Contact Info

Product

Resources

About