Disentangled Representations for Manipulation of Sentiment in Text

Larsson, Maria; Nilsson, Amanda; Kågebäck, Mikael

doi:10.48550/arxiv.1712.10066

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…searchers use different methods to measure this preservation of semantics. Despite its disadvantages (Larsson, Nilsson, and Kågebäck 2017), one of the most widely used semantic similarity metrics is BLEU. (Tikhonov et al 2019) show that it could be manipulated in a way that the system would show higher values of BLEU on average, producing sentences that are completely detached from the input semantically.…”

Section: Measuring Semantic Preservationmentioning

confidence: 99%

Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metric

Yamshchikov¹,

Shibaev²,

Khlebnikov³

et al. 2021

AAAI

View full text Add to dashboard Cite

The rapid development of such natural language processing tasks as style transfer, paraphrase, and machine translation often calls for the use of semantic similarity metrics. In recent years a lot of methods to measure the semantic similarity of two short texts were developed. This paper provides a comprehensive analysis for more than a dozen of such methods. Using a new dataset of fourteen thousand sentence pairs human-labeled according to their semantic similarity, we demonstrate that none of the metrics widely used in the literature is close enough to human judgment in these tasks. A number of recently proposed metrics provide comparable results, yet Word Mover Distance is shown to be the most reasonable solution to measure semantic similarity in reformulated texts at the moment.

show abstract

Section: Measuring Semantic Preservationmentioning

confidence: 99%

Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metric

Yamshchikov¹,

Shibaev²,

Khlebnikov³

et al. 2021

AAAI

View full text Add to dashboard Cite

show abstract

“…Compared to vision, there has been relatively little work on learning disentangled representations of text. Much of the prior work on disentanglement for NLP that does exist has focused on using such representations to facilitate controlled generation, e.g., manipulating sentiment (Larsson et al, 2017).…”

Section: Related Workmentioning

confidence: 99%

Disentangling Representations of Text by Masking Transformers

Zhang

Meent

Wallace

2021

Preprint

View full text Add to dashboard Cite

Representations from large pretrained models such as BERT encode a range of features into monolithic vectors, affording strong predictive accuracy across a multitude of downstream tasks. In this paper we explore whether it is possible to learn disentangled representations by identifying existing subnetworks within pretrained models that encode distinct, complementary aspect representations. Concretely, we learn binary masks over transformer weights or hidden units to uncover subsets of features that correlate with a specific factor of variation; this eliminates the need to train a disentangled model from scratch for a particular task. We evaluate this method with respect to its ability to disentangle representations of sentiment from genre in movie reviews, "toxicity" from dialect in Tweets, and syntax from semantics.By combining masking with magnitude pruning we find that we can identify sparse subnetworks within BERT that strongly encode particular aspects (e.g., toxicity) while only weakly encoding others (e.g., race). Moreover, despite only learning masks, we find that disentanglement-viamasking performs as well as -and often better than -previously proposed methods based on variational autoencoders and adversarial training.

show abstract

“…Other recent work has focused on text generation from factorized representations (Larsson et al, 2017). And Zhang et al (2017) proposed a lightly supervised method for domain adaptation using aspect-augmented neural networks.…”

Section: Related Workmentioning

confidence: 99%

Learning Disentangled Representations of Texts with Application to Biomedical Abstracts

Jain¹,

Banner²,

Meent³

et al. 2018

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

We propose a method for learning disentangled representations of texts that code for distinct and complementary aspects, with the aim of affording efficient model transfer and interpretability. To induce disentangled embeddings, we propose an adversarial objective based on the (dis)similarity between triplets of documents with respect to specific aspects. Our motivating application is embedding biomedical abstracts describing clinical trials in a manner that disentangles the populations, interventions, and outcomes in a given trial. We show that our method learns representations that encode these clinically salient aspects, and that these can be effectively used to perform aspect-specific retrieval. We demonstrate that the approach generalizes beyond our motivating application in experiments on two multi-aspect review corpora.

show abstract

Disentangled Representations for Manipulation of Sentiment in Text

Cited by 3 publications

References 0 publications

Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metric

Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metric

Disentangling Representations of Text by Masking Transformers

Learning Disentangled Representations of Texts with Application to Biomedical Abstracts

Contact Info

Product

Resources

About