Negative Lexically Constrained Decoding for Paraphrase Generation

Kajiwara, Tomoyuki

doi:10.18653/v1/p19-1607

Cited by 28 publications

(28 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chen et al (2019) addressed the same problem, but the syntax is controlled by a sentence exemplar. Kajiwara (2019) proposed a model that first identifies a set of words to be paraphrased, and then generates the output by using a pre-trained paraphrase generation model. proposed a Transformer-based model that utilizes structured semantic knowledge to improve the quality of paraphrases.…”

Section: Related Workmentioning

confidence: 99%

Paraphrase Generation by Learning How to Edit from Samples

Amirhossein¹,

Salehi²,

Baghshah³

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Neural sequence to sequence text generation has been proved to be a viable approach to paraphrase generation. Despite promising results, paraphrases generated by these models mostly suffer from lack of quality and diversity. To address these problems, we propose a novel retrieval-based method for paraphrase generation. Our model first retrieves a paraphrase pair similar to the input sentence from a pre-defined index. With its novel editor module, the model then paraphrases the input sequence by editing it using the extracted relations between the retrieved pair of sentences. In order to have fine-grained control over the editing process, our model uses the newly introduced concept of Micro Edit Vectors. It both extracts and exploits these vectors using the attention mechanism in the Transformer architecture. Experimental results show the superiority of our paraphrase generation method in terms of both automatic metrics, and human evaluation of relevance, grammaticality, and diversity of generated paraphrases.

show abstract

Section: Related Workmentioning

confidence: 99%

Paraphrase Generation by Learning How to Edit from Samples

Amirhossein¹,

Salehi²,

Baghshah³

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Seq2seq-based methods have been widely used in the task of paraphrase generation (Prakash et al, 2016;Kajiwara, 2019). Li et al (2018) further adopt reinforcement learning with policy gradient technique to generate semantically consistent paraphrases.…”

Section: Paraphrase Generationmentioning

confidence: 99%

“…Early works on paraphrase generation mainly focus on rule-based (McKeown, 1983;Meteer and Shaked, 1988), grammar-based (Narayan et al, 2016), lexicon-based (Bolshakov and Gelbukh, 2004;Kauchak and Barzilay, 2006), and statistical machine translation (SMT)-based methods (Kauchak and Barzilay, 2006;Zhao et al, 2009). Recently, with the release of large-scale paraphrase datasets, sequence-to-sequence (seq2seq) models (Prakash et al, 2016;Kajiwara, 2019;Li et al, 2018;Gupta et al, 2018;Shakeri and Sethy, 2019;Yang et al, 2019) have become the dominant technique in the field of paraphrase generation.…”

Section: Introductionmentioning

confidence: 99%

DivGAN: Towards Diverse Paraphrase Generation via Diversified Generative Adversarial Network

Cao¹,

Wan²

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Paraphrases refer to texts that convey the same meaning with different expression forms. Traditional seq2seq-based models on paraphrase generation mainly focus on the fidelity while ignoring the diversity of outputs. In this paper, we propose a deep generative model to generate diverse paraphrases. We build our model based on the conditional generative adversarial network, and propose to incorporate a simple yet effective diversity loss term into the model in order to improve the diversity of outputs. The proposed diversity loss maximizes the ratio of pairwise distance between the generated texts and their corresponding latent codes, forcing the generator to focus more on the latent codes and produce diverse samples. Experimental results on benchmarks of paraphrase generation show that our proposed model can generate more diverse paraphrases compared with baselines.

show abstract

“…On the other hand, as pointed in Kajiwara (2019), paraphrase generation rewrites only a limited portion of an original input and the reference often includes some words occurred in the original input, thus a sentence with higher ROUGE-ref score may have low diversity (Miao et al, 2019). Therefore, the reward based on reference do not focus on the variations between the sampled sentence and the original input.…”

Section: Rewards For Multi-objective Learningmentioning

confidence: 99%

“…In recent years, there are also growing interests in generating lexically and syntactically diverse paraphrases (Gupta et al, 2018;Xu et al, 2018b;Xu et al, 2018a;Park et al, 2019;Qian et al, 2019;Kajiwara, 2019). For Seq2Seq models, the techniques of generating diverse paraphrases mainly include two categories: i) applying decoding methods such as using beam search or multiple decoders; ii) introducing random noise as model input.…”

Section: Introductionmentioning

confidence: 99%

A Learning-Exploring Method to Generate Diverse Paraphrases with Multi-Objective Deep Reinforcement Learning

Liu

Yang

Xiong

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

Paraphrase generation (PG) is of great importance to many downstream tasks in natural language processing. Diversity is an essential nature to PG for enhancing generalization capability and robustness of downstream applications. Recently, neural sequence-to-sequence (Seq2Seq) models have shown promising results in PG. However, traditional model training for PG focuses on optimizing model prediction against single reference and employs cross-entropy loss, which objective is unable to encourage model to generate diverse paraphrases. In this work, we present a novel approach with multi-objective learning to PG. We propose a learning-exploring method to generate sentences as learning objectives from the learned data distribution, and employ reinforcement learning to combine these new learning objectives for model training. We first design a sample-based algorithm to explore diverse sentences. Then we introduce several reward functions to evaluate the sampled sentences as learning signals in terms of expressive diversity and semantic fidelity, aiming to generate diverse and high-quality paraphrases. To effectively optimize model performance satisfying different evaluating aspects, we use a GradNorm-based algorithm that automatically balances these training objectives. Experiments and analyses on Quora and Twitter datasets demonstrate that our proposed method not only gains a significant increase in diversity but also improves generation quality over several state-of-the-art baselines.

show abstract

Negative Lexically Constrained Decoding for Paraphrase Generation

Cited by 28 publications

References 19 publications

Paraphrase Generation by Learning How to Edit from Samples

Paraphrase Generation by Learning How to Edit from Samples

DivGAN: Towards Diverse Paraphrase Generation via Diversified Generative Adversarial Network

A Learning-Exploring Method to Generate Diverse Paraphrases with Multi-Objective Deep Reinforcement Learning

Contact Info

Product

Resources

About