ConRPG: Paraphrase Generation using Contexts as Regularizer

Meng, Yuxian; Ao, Xiang; He, Qianhua; Sun, Xiaofei; Han, Qinghong; Wu, Fei; Fan, Chun; Li, Jiwei

doi:10.48550/arxiv.2109.00363

Cited by 4 publications

(6 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Paraphrases express the surface forms of the underlying semantic content [6] and capture the essence of language diversity [46]. Early work on automatic generation of paraphrase are generally rule-based [7,8], but the recent trend brings to fore neural network solutions [47,19,10,12,13,20,6]. Current research for paraphrasing mainly focuses on supervised methods, which require the availability of a large number of source and target pairs.…”

Section: Paraphrase Generationmentioning

confidence: 99%

See 1 more Smart Citation

Deep Latent Variable Models for Semi-supervised Paraphrase Generation

Yu¹,

Cristea²,

Harit³

et al. 2023

Preprint

View full text Add to dashboard Cite

This paper explores deep latent variable models for semi-supervised paraphrase generation, where the missing target pair is modelled as a latent paraphrase sequence. We present a novel unsupervised model named variational sequence auto-encoding reconstruction (VSAR), which performs latent sequence inference given an observed text. To leverage information from text pairs, we introduce a supervised model named dual directional learning (DDL). Combining VSAR with DDL (DDL+VSAR) enables us to conduct semi-supervised learning; however, the combined model suffers from a coldstart problem. To combat this issue, we propose to deal with better weight initialisation, leading to a two-stage training scheme named knowledge reinforced training. Our empirical evaluations suggest that the combined model yields competitive performance against the state-of-the-art supervised baselines on complete data. Furthermore, in scenarios where only a fraction of the labelled pairs are available, our combined model consistently outperforms the strong supervised model baseline (DDL) by a significant margin.

show abstract

Section: Paraphrase Generationmentioning

confidence: 99%

“…Traditional solutions to paraphrase generation are generally rule-based [7,8], utilising lexical resources, such as WordNet [9], to find word replacements. The recent trend brings to fore neural network models [10,11,12,13], which are typically based on a sequence-to-sequence learning paradigm [14].…”

Section: Introductionmentioning

confidence: 99%

Deep Latent Variable Models for Semi-supervised Paraphrase Generation

Yu¹,

Cristea²,

Harit³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…An ideal paraphrase not only needs to have the same semantics but also should have a significant change in expression from the input sentence (i.e., expression difference) (Bhagat and Hovy, 2013). Aiming at the problem of expression differences in generated sentences, researchers have made a lot of attempts in different dimensions (Lin and Wan, 2021;Li et al, 2019;Hosking and Lapata, 2021;Meng et al, 2021). For example, Li et al proposed multiple generators with different granularity levels to learn the mapping relationship between input and output respectively, and then combine them to complete the paraphrase generation task (Li et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

Vector-Quantized Prompt Learning for Paraphrase Generation

Luo,

Liu,

Liu

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Deep generative modeling of natural languages has achieved many successes, such as producing fluent sentences and translating from one language into another. However, the development of generative modeling techniques for paraphrase generation still lags behind largely due to the challenges in addressing the complex conflicts between expression diversity and semantic preservation. This paper proposes to generate diverse and high-quality paraphrases by exploiting the pre-trained models with instance-dependent prompts. To learn generalizable prompts, we assume that the number of abstract transforming patterns of paraphrase generation (governed by prompts) is finite and usually not large. Therefore, we present vector-quantized prompts as the cues to control the generation of pre-trained models. Extensive experiments demonstrate that the proposed method achieves new state-of-art results on three benchmark datasets, including Quora, Wikianswers, and MSCOCO. We will release all the code upon acceptance.

show abstract

“…Other unsupervised methods for paraphrase generation include VAE (VQ-VAE) (Roy and Grangier, 2019), latent bagof-words alignment (Fu et al, 2019) and simulated annealing (Liu et al, 2019a). Adapting large-scale pretraining (Devlin et al, 2018;Radford et al, 2018;Liu et al, 2019b;Clark et al, 2020;Sun et al, 2021b) to paraphrase generation has been recently investigated (Witteveen and Andrews, 2019;Hegde and Patil, 2020;Niu et al, 2020;Meng et al, 2021) and has shown promising potentials to improve generation quality. Our work is distantly related to unsupervised text style transfer (Hu et al, 2017;Mueller et al, 2017;Shen et al, 2017;Li et al, 2018a;Fu et al, 2018), where the model alters a specific text attribute of an input sentence (such as sentiment) while preserving other text attributes.…”

Section: Related Workmentioning

confidence: 99%

Paraphrase Generation as Unsupervised Machine Translation

Fan¹,

Tian²,

Meng³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a new paradigm for paraphrase generation by treating the task as unsupervised machine translation (UMT) based on the assumption that there must be pairs of sentences expressing the same meaning in a large-scale unlabeled monolingual corpus. The proposed paradigm first splits a large unlabeled corpus into multiple clusters, and trains multiple UMT models using pairs of these clusters. Then based on the paraphrase pairs produced by these UMT models, a unified surrogate model can be trained to serve as the final SEQ2SEQ model to generate paraphrases, which can be directly used for test in the unsupervised setup, or be finetuned on labeled datasets in the supervised setup. The proposed method offers merits over machinetranslation-based paraphrase generation methods, as it avoids reliance on bilingual sentence pairs. It also allows human intervene with the model so that more diverse paraphrases can be generated using different filtering criteria. Extensive experiments on existing paraphrase dataset for both the supervised and unsupervised setups demonstrate the effectiveness the proposed paradigm.

show abstract

ConRPG: Paraphrase Generation using Contexts as Regularizer

Cited by 4 publications

References 54 publications

Deep Latent Variable Models for Semi-supervised Paraphrase Generation

Deep Latent Variable Models for Semi-supervised Paraphrase Generation

Vector-Quantized Prompt Learning for Paraphrase Generation

Paraphrase Generation as Unsupervised Machine Translation

Contact Info

Product

Resources

About