Improving Formality Style Transfer with Context-Aware Rule Injection

Yao, Zonghai; Yu, Hong

doi:10.18653/v1/2021.acl-long.124

Cited by 7 publications

(10 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous studies (Rao and Tetreault, 2018;Niu et al, 2018; typically train seq2seq encoder-decoder models on this benchmark. Recent studies (Wang et al, 2019;Yao and Yu, 2021;Chawla and Yang, 2020;Lai et al, 2021) have deduced that fine-tuning large-scale pretrained models such as GPT-2 (Radford et al, 2019) and BART (Lewis et al, 2020) on the parallel corpora can improve the performance. To address the data-scarcity problem of parallel datasets, proposed three data augmentation techniques to augment pseudo-parallel data for training.…”

Section: Related Workmentioning

confidence: 99%

“…There are some typical informal expressions in the parallel corpus, such as the use of slang words and abbreviations, capitalized words for emphasis, and spelling errors. Some existing studies (Wang et al, 2019;Yao and Yu, 2021) adopt editing rules to revise such informal expressions as a preprocessing step. Inspired by these, we propose the adoption of opposite rules to synthesize such noises.…”

Section: Data Perturbation Strategiesmentioning

confidence: 99%

“…• CARI (Yao and Yu, 2021) improves GPT-CAT by using BERT (Devlin et al, 2018) to select optimal rules to pre-process the informal sentences.…”

Section: Comparison With Previous Workmentioning

confidence: 99%

“…The large amount of usergenerated data from online resources like tweets often contain informal expressions such as slang words (e.g., gonna), wrong capitalization or punctuations, and grammatical or spelling errors. FST can clean and formalize such noisy data, to benefit downstream NLP applications such as sentiment classification (Yao and Yu, 2021). Some examples of FST data are presented in Table 1.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Semi-Supervised Formality Style Transfer with Consistency Training

Liu¹,

Wang²,

Okazaki³

2022

Preprint

View full text Add to dashboard Cite

Formality style transfer (FST) is a task that involves paraphrasing an informal sentence into a formal one without altering its meaning. To address the data-scarcity problem of existing parallel datasets, previous studies tend to adopt a cycle-reconstruction scheme to utilize additional unlabeled data, where the FST model mainly benefits from target-side unlabeled sentences. In this work, we propose a simple yet effective semi-supervised framework to better utilize source-side unlabeled sentences based on consistency training. Specifically, our approach augments pseudoparallel data obtained from a source-side informal sentence by enforcing the model to generate similar outputs for its perturbed version. Moreover, we empirically examined the effects of various data perturbation methods and propose effective data filtering strategies to improve our framework. Experimental results on the GYAFC benchmark demonstrate that our approach can achieve state-of-the-art results, even with less than 40% of the parallel data 1 .

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Data Perturbation Strategiesmentioning

confidence: 99%

“…• CARI (Yao and Yu, 2021) improves GPT-CAT by using BERT (Devlin et al, 2018) to select optimal rules to pre-process the informal sentences.…”

Section: Comparison With Previous Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Semi-Supervised Formality Style Transfer with Consistency Training

Liu¹,

Wang²,

Okazaki³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To mitigate the consequent problem of noisy parallel data, the encoder was presented with an input which was a concatenation of the original informal sentence and its formal revision. Yao and Yu (2021) explored a similar architecture. The encoder's input was created by concatenating the original sentence and some supplementary information, which comprised an exhaustive list of all matched rules and the corresponding text alternatives, arranged as tuples.…”

Section: Formalitymentioning

confidence: 99%

From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey

Troiano¹,

Aswathy²,

Klinger³

2021

Preprint

View full text Add to dashboard Cite

Humans are naturally endowed with the ability to write in a particular style. They can, for instance, rephrase a formal letter in an informal way, convey a literal message with the use of figures of speech, edit a novel mimicking the style of some well-known authors. Automating this form of creativity constitutes the goal of style transfer. As a natural language generation task, style transfer aims at re-writing existing texts, and specifically, it creates paraphrases that exhibit some desired stylistic attributes. From a practical perspective, it envisions beneficial applications, like chat-bots that modulate their communicative style to appear empathetic, or systems that automatically simplify technical articles for a non-expert audience. Style transfer has been dedicated several style-aware paraphrasing methods. A handful of surveys give a methodological overview of the field, but they do not support researchers to focus on specific styles. With this paper, we aim at providing a comprehensive discussion of the styles that have received attention in the transfer task. We organize them into a hierarchy, highlighting the challenges for the definition of each of them, and pointing out gaps in the current research landscape. The hierarchy comprises two main groups. One encompasses styles that people modulate arbitrarily, along the lines of registers and genres. The other group corresponds to unintentionally expressed styles, due to an author's personal characteristics. Hence, our review shows how the groups relate to one another, and where specific styles, including some that have never been explored, belong in the hierarchy. Moreover, we summarize the methods employed for different stylistic families, hinting researchers towards those that would be the most fitting for future research.

show abstract

From theories on styles to their transfer in text: Bridging the gap with a hierarchical survey

2022

View full text Add to dashboard Cite

Humans are naturally endowed with the ability to write in a particular style. They can, for instance, rephrase a formal letter in an informal way, convey a literal message with the use of figures of speech or edit a novel by mimicking the style of some well-known authors. Automating this form of creativity constitutes the goal of style transfer. As a natural language generation task, style transfer aims at rewriting existing texts, and specifically, it creates paraphrases that exhibit some desired stylistic attributes. From a practical perspective, it envisions beneficial applications, like chatbots that modulate their communicative style to appear empathetic, or systems that automatically simplify technical articles for a non-expert audience. Several style-aware paraphrasing methods have attempted to tackle style transfer. A handful of surveys give a methodological overview of the field, but they do not support researchers to focus on specific styles. With this paper, we aim at providing a comprehensive discussion of the styles that have received attention in the transfer task. We organize them in a hierarchy, highlighting the challenges for the definition of each of them and pointing out gaps in the current research landscape. The hierarchy comprises two main groups. One encompasses styles that people modulate arbitrarily, along the lines of registers and genres. The other group corresponds to unintentionally expressed styles, due to an author’s personal characteristics. Hence, our review shows how these groups relate to one another and where specific styles, including some that have not yet been explored, belong in the hierarchy. Moreover, we summarize the methods employed for different stylistic families, hinting researchers towards those that would be the most fitting for future research.

show abstract

Improving Formality Style Transfer with Context-Aware Rule Injection

Cited by 7 publications

References 31 publications

Semi-Supervised Formality Style Transfer with Consistency Training

Semi-Supervised Formality Style Transfer with Consistency Training

From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey

From theories on styles to their transfer in text: Bridging the gap with a hierarchical survey

Contact Info

Product

Resources

About