A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer

Abstract

Unsupervised text style transfer aims to transfer the underlying style of text but keep its main content unchanged without parallel data. Most existing methods typically follow two steps: first separating the content from the original style, and then fusing the content with the desired style. However, the separation in the first step is challenging because the content and style interact in subtle ways in natural language. Therefore, in this paper, we propose a dual reinforcement learning framework to directly transfer the style of the text via a one-step mapping model, without any separation of content and style. Specifically, we consider the learning of the source-to-target and targetto-source mappings as a dual task, and two rewards are designed based on such a dual structure to reflect the style accuracy and content preservation, respectively. In this way, the two one-step mapping models can be trained via reinforcement learning, without any use of parallel data. Automatic evaluations show that our model outperforms the state-of-the-art systems by a large margin, especially with more than 8 BLEU points improvement averaged on two benchmark datasets. Human evaluations also validate the effectiveness of our model in terms of style accuracy, content preservation and fluency. Our code and data, including outputs of all baselines and our model are available at https://github.com/luofuli/DualLanST. 1

Reformulating Unsupervised Style Transfer as Paraphrase Generation

“…Our approach requires none of the finicky 2 modeling paradigms popular in style transfer researchno reinforcement learning (Luo et al, 2019), variational inference (He et al, 2020), or autoregressive sampling during training (Subramanian et al, 2019). Instead, we implement the first two stages of our pipeline by simply fine-tuning a pretrained GPT-2 language model .…”

“…The original code of Subramanian et al (2019) has not been open-sourced. 13 Results with other metrics such as BLEU, as well as comparisons against several other baselines like ; Prabhumoye et al ( 2018); Luo et al (2019); ; Sudhakar et al (2019) Table 3: Ablation study using automatic metrics on the Formality (Form.) and Shakespeare (Shak.)…”

“…Unsupervised style transfer is often modeled by disentangling style & content using attribute classifiers (Hu et al, 2017;Shen et al, 2017), policy gradient training (Xu et al, 2018;Luo et al, 2019) or retrieval-based approaches . Recently, backtranslation has emerged as a method to model semantic preservation (Prabhumoye et al, 19 We could not find an easy way to perform 11-way style transfer in the baseline models without significantly modifying their codebase / model due to the complex probabilistic formulation beyond 2 styles and separate modeling for each of the 110 directions.…”

“…We present more comparisons with prior work in Table 13. We use the generated outputs for the Formality test set available in the public repository of Luo et al (2019) (including outputs from the algorithms described in Prabhumoye et al, 2018 and and run them on our evaluation pipeline. We compare the results with our formality transfer model used in Table 1 and Table 2.…”

Rethinking Text Attribute Transfer: A Lexical Analysis

“…Because of the lack of parallel datasets, most models focus on the unpaired transfer. Although plenty of sophisticated techniques are used in this task, such as adversarial learning Chen et al, 2018), latent representations (Li and Mandt, 2018;Dai et al, 2019;Liu et al, 2019), and reinforcement learning (Luo et al, 2019;Gong et al, 2019;Xu et al, 2018), there is little discussion about what is changed and what remains unchanged.…”