Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer

Abstract

We consider the task of text attribute transfer: transforming a sentence to alter a specific attribute (e.g., sentiment) while preserving its attribute-independent content (e.g., changing "screen is just the right size" to "screen is too small"). Our training data includes only sentences labeled with their attribute (e.g., positive or negative), but not pairs of sentences that differ only in their attributes, so we must learn to disentangle attributes from attributeindependent content in an unsupervised way. Previous work using adversarial methods has struggled to produce high-quality outputs. In this paper, we propose simpler methods motivated by the observation that text attributes are often marked by distinctive phrases (e.g., "too small"). Our strongest method extracts content words by deleting phrases associated with the sentence's original attribute value, retrieves new phrases associated with the target attribute, and uses a neural model to fluently combine these into a final output. On human evaluation, our best method generates grammatical and appropriate responses on 22% more inputs than the best previous system, averaged over three attribute transfer datasets: altering sentiment of reviews on Yelp, altering sentiment of reviews on Amazon, and altering image captions to be more romantic or humorous.

Evaluating Style Transfer for Text

“…To evaluate content preservation between x and x , previous work has used BLEU Li et al, 2018), an n-gram based metric originally designed to evaluate machine translation models (Papineni et al, 2002). BLEU does not take into account the aim of style transfer models, which is to alter style by necessarily changing words.…”

“…(Table 2 shows sample words in the lexicon constructed for the dataset used in our experiments.) While sentiment datasets have been widely used in the literature (Shen et al, 2017;Li et al, 2018), a lexicon can be constructed for other datasets in the same manner, as long as the dataset has style labels.…”

“…It identifies and removes style words from texts, searches for related words pertaining to a new target style, and combines the de-stylized text with the search results using a neural model. We train DAR on γ = 15, where γ is a threshold parameter for the maximum number of style words that can be removed from texts, with (Li et al, 2018).…”

Weakly-Supervised Deep Learning for Domain Invariant Sentiment Classification

“…However, we witness models trained on reviews of a particular domain, do not generally do well when tested on reviews of the unknown and different target domain. The several limitations in the textual domain transfer primarily exist due to out-of-the-vocabulary tokens [19], stylistic variations [16], nongeneralizable features [12], etc. For example, Crammer et al [5] assumed that the distributions of multiple sources are the same, but the labelings of the data from different sources may be different from each other.…”

ALTER: Auxiliary Text Rewriting Tool for Natural Language Generation

“…The benefits of revision histories are three-fold. Firstly, revision histories can provide supervision signals for the generative models, which consider rewriting as applying a sequence of edit operations on text (Li et al, 2018;Guu et al, 2018). Secondly, revision histories can potentially provide deep insights regarding cognitive process and human edit behaviours in varying demographic groups.…”