Reinforced Rewards Framework for Text Style Transfer

Abstract

Style transfer deals with the algorithms to transfer the stylistic properties of a piece of text into that of another while ensuring that the core content is preserved. There has been a lot of interest in the field of text style transfer due to its wide application to tailored text generation. Existing works evaluate the style transfer models based on content preservation and transfer strength. In this work, we propose a reinforcement learning based framework that directly rewards the framework on these target metrics yielding a better transfer of the target style. We show the improved performance of our proposed framework based on automatic and human evaluation on three independent tasks: wherein we transfer the style of text from formal to informal, high excitement to low excitement, modern English to Shakespearean English, and vice-versa in all the three cases. Improved performance of the proposed framework over existing state-of-the-art frameworks indicates the viability of the approach.

Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer

“…Evaluation Following previous work He et al, 2020;Sancheti et al, 2020), we adopt the following strategies. The binary classifier TextCNN (Kim, 2014) is pre-trained to evaluate style strength; on the human references it has an accuracy of 87.0% (E&M) and 89.3% (F&R).…”

“…For the GPT-2 based model, we also add a classification confidence reward to the source sentence, similar to Eq. 4, since the model generates sentence x with the original style while generating the target sentence: BLEU Score Reward Following Sancheti et al (2020), we introduce a BLEU-based reward to foster content preservation as in Eq. 6, where y is the target style text obtained by greedily maximizing the distribution of model outputs at each time step, and y s is sampled from the distribution.…”

“…Niu et al (2018) combine the train sets of two different domains and incorporate machine translation to train their models with a multi-task learning schema, plus model ensembles. Sancheti et al (2020) use it to train a supervised sequence-tosequence model, and in addition to the commonly used style strength reward, they include a reward based on BLEU (Papineni et al, 2002) to enhance content preservation. Shang et al (2019) propose a semi-supervised model combining parallel data with large amounts of non-parallel data.…”

Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus

“…Recent works have also modeled this in a self-supervised manner where rewriting (transfer) is achieved by utilizing corpus from the target style alone (Syed et al, 2020). These wide studies have also led to the curation and benchmarking of non-parallel dataset for various style dimensions, such as sentiment (Li et al, 2018), formality (Rao and Tetreault, 2018), politeness (Danescu-Niculescu-Mizil et al, 2013), excitement (Sancheti et al, 2020), etc. But availability of data with joint tagging across multiple styles is limited and has restricted the ability of existing approaches to scale from single-dimensional transfer to multiple style dimensions.…”

Human Perception in Natural Language Generation

“…We then fine-tune GePpeTto with this perceptionlabelled data. In addition, inspired by the classifierbased reward used in style transfer tasks (Lample et al, 2019;Gong et al, 2019;Luo et al, 2019;Sancheti et al, 2020), we reward the model to push its classification confidence. We evaluate the new perception-enhanced models in comparison with the original GePpeTto by running both an automatic as well as a human evaluation on output generated by the various models.…”