Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer

Li, Juncen; Jia, Robin; He, He; Liang, Percy

doi:10.48550/arxiv.1804.06437

Cited by 26 publications

(41 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our work, we leverage such a lexical resource for connotations (Allaway and McKeown, 2020) to reframe arguments to be more trustworthy (e.g., less partisan, no appeal to fear fallacy), while maintaining the same denotative meaning. While retrieve-and-replace methods perform well on other attribute transfer tasks such as sentiment (Li et al, 2018a;Sudhakar et al, 2019a), our task is more dependent on broader context within a sentence even though we are performing localized replacement. Thus, there are two main challenges we need to address: 1) the lack of a parallel dataset of negatively and positively framed arguments (naturallyoccurring); and 2) a generation approach that can not only change the connotative meaning but also keep the same denotative meaning of the input argument.…”

Section: Arg4mentioning

confidence: 99%

“…Moreover, for lexical framing, subtle differences in word choices matter the most. By explicitly using special tokens ([SEP]) in our parallel data during fine-tuning, the BART model learns what to edit, instead of editing random words in the sentence, a common issue often found in attribute transfer models (Li et al, 2018a;Sudhakar et al, 2019a). At test time, therefore, we can ensure the model reframes a desired content span.…”

Section: Controllable Text Generationmentioning

confidence: 99%

“…Generative Style Transformer (GST) : We use the state of art for text style transfer by Sudhakar et al (2019a), which is a part of a larger "Delete Retrieve Generate" framework (Li et al, 2018a). To maintain parity with other baselines, instead of letting the model delete attribute keywords we delete the partisan collocations or fear related words from the arguments as the first step, followed by the usual retrieve and generate steps.…”

Section: Lexical Replacement (Lexrep)mentioning

confidence: 99%

See 2 more Smart Citations

ENTRUST: Argument Reframing with Language Models and Entailment

Chakrabarty¹,

Hidey²,

Muresan³

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Framing" involves the positive or negative presentation of an argument or issue depending on the audience and goal of the speaker (Entman, 1983). Differences in lexical framing, the focus of our work, can have large effects on peoples' opinions and beliefs. To make progress towards reframing arguments for positive effects, we create a dataset and method for this task. We use a lexical resource for "connotations" to create a parallel corpus and propose a method for argument reframing that combines controllable text generation (positive connotation) with a postdecoding entailment component (same denotation). Our results show that our method is effective compared to strong baselines along the dimensions of fluency, meaning, and trustworthiness/reduction of fear.

show abstract

Section: Arg4mentioning

confidence: 99%

Section: Controllable Text Generationmentioning

confidence: 99%

Section: Lexical Replacement (Lexrep)mentioning

confidence: 99%

See 1 more Smart Citation

ENTRUST: Argument Reframing with Language Models and Entailment

Chakrabarty¹,

Hidey²,

Muresan³

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…We use the Yelp reviews dataset collected by Shen et al (2017) which contains 250K negative sentences and 380K positive sentences. We also use a small test set that has 1000 human-annotated parallel sentences introduced in Li et al (2018). We denote the positive sentiment as domain D 1 and the negative sentiment as domain D 2 .…”

Section: Datasets and Experiments Setupmentioning

confidence: 99%

A Probabilistic Formulation of Unsupervised Text Style Transfer

He,

Wang,

Neubig

et al. 2020

Preprint

View full text Add to dashboard Cite

We present a deep generative model for unsupervised text style transfer that unifies previously proposed non-generative techniques. Our probabilistic approach models non-parallel data from two domains as a partially observed parallel corpus. By hypothesizing a parallel latent sequence that generates each observed sequence, our model learns to transform sequences from one domain to another in a completely unsupervised fashion. In contrast with traditional generative sequence models (e.g. the HMM), our model makes few assumptions about the data it generates: it uses a recurrent language model as a prior and an encoder-decoder as a transduction distribution. While computation of marginal data likelihood is intractable in this model class, we show that amortized variational inference admits a practical surrogate. Further, by drawing connections between our variational objective and other recent unsupervised style transfer and machine translation techniques, we show how our probabilistic view can unify some known non-generative objectives such as backtranslation and adversarial loss. Finally, we demonstrate the effectiveness of our method on a wide range of unsupervised style transfer tasks, including sentiment transfer, formality transfer, word decipherment, author imitation, and related language translation. Across all style transfer tasks, our approach yields substantial gains over state-of-the-art non-generative baselines, including the state-of-the-art unsupervised machine translation techniques that our approach generalizes. Further, we conduct experiments on a standard unsupervised machine translation task and find that our unified approach matches the current state-of-the-art. 1 * Equal Contribution. 1 Code and data are available at https://github.com/cindyxinyiwang/deep-latent-sequence-model. 2 Notably, some tasks we evaluate on do change content to some degree, such as sentiment transfer, but for conciseness we use the term "style transfer" nonetheless.

show abstract

“…Recent work has explored a variety text generation tasks that condition on a control variable to specify a desired trait of the output. Examples include summarization conditioned on a desired output length (Fan et al, 2018), paraphrase generation conditioned on a parse tree (Krishna et al, 2020), style transfer conditioned on sentiment (He et al, 2020b), and more (Hu et al, 2017;Li et al, 2018;Fu et al, 2018;He et al, 2020a). In this work, we specifically focus on text generation tasks that condition on a scalar control variable, as depicted in Figure 1.…”

Section: Introductionmentioning

confidence: 99%

An Empirical Study of Extrapolation in Text Generation with Scalar Control

Jain¹,

Berg-Kirkpatrick²

2021

Preprint

View full text Add to dashboard Cite

We conduct an empirical evaluation of extrapolation performance when conditioning on scalar control inputs like desired output length, desired edit from an input sentence, and desired sentiment across three text generation tasks. Specifically, we examine a zero-shot setting where models are asked to generalize to ranges of control values not seen during training. We focus on evaluating popular embedding methods for scalar inputs, including both learnable and sinusoidal embeddings, as well as simpler approaches. Surprisingly, our findings indicate that the simplest strategy of using scalar inputs directly, without further encoding, most reliably allows for successful extrapolation. Related WorkKikuchi et al. (2016) and Liu et al. (2018) modified the initial state of the decoder in seq2seq models in order to condition on control values. Sennrich et al.

show abstract

Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer

Cited by 26 publications

References 10 publications

ENTRUST: Argument Reframing with Language Models and Entailment

ENTRUST: Argument Reframing with Language Models and Entailment

A Probabilistic Formulation of Unsupervised Text Style Transfer

An Empirical Study of Extrapolation in Text Generation with Scalar Control

Contact Info

Product

Resources

About