Response Generation by Context-Aware Prototype Editing

Abstract

Open domain response generation has achieved remarkable progress in recent years, but sometimes yields short and uninformative responses. We propose a new paradigm, prototypethen-edit for response generation, that first retrieves a prototype response from a pre-defined index and then edits the prototype response according to the differences between the prototype context and current context. Our motivation is that the retrieved prototype provides a good start-point for generation because it is grammatical and informative, and the post-editing process further improves the relevance and coherence of the prototype. In practice, we design a contextaware editing model that is built upon an encoder-decoder framework augmented with an editing vector. We first generate an edit vector by considering lexical differences between a prototype context and current context. After that, the edit vector and the prototype response representation are fed to a decoder to generate a new response. Experiment results on a large scale dataset demonstrate that our new paradigm significantly increases the relevance, diversity and originality of generation results, compared to traditional generative models. Furthermore, our model outperforms retrieval-based methods in terms of relevance and originality.

Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing

“…Retrieveand-edit uses a copying mechanism to copy tokens from the retrieved example . Edit vector calculates an edit vector by considering lexical differences between a prototype context and current context, and uses the edit vector as an extra feature (Wu et al, 2018). We can see that applying the MAML framework to the Seq2Action model achieves a gain of 1.35% exact match accuracy.…”

“…There are recent attempts at exploiting retrieved examples to improve the generation of logical form and text. Retrieve-and-edit approaches Huang et al, 2018b;Wu et al, 2018;Gu et al, 2017) typically first use a context-independent retriever to find the most relevant datapoint, and then use it as an additional input of the editing model. However, a contextaware retriever is very important for the task of context-dependent semantic parsing.…”

MoEL: Mixture of Empathetic Listeners

“…Neural network approaches for conversation models have shown to be successful in scalable training and generating fluent and relevant responses (Vinyals and Le, 2015). However, it has been pointed out by Li et al (2016a,b,c); Wu et al (2018b) that only using Maximum Likelihood Estimation as the objective function tends to lead to generic and repetitive responses like "I am sorry". Furthermore, many others have shown that the incorporation of additional inductive bias leads to a more engaging chatbot, such as understanding commonsense (Dinan et al, 2018), or modeling consistent persona (Li et al, 2016b;Zhang et al, 2018a;Mazare et al, 2018a).…”

Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation

“…Our work is also related to the recent work on applying retrieval mechanisms to augment text generation, such as image captioning (Kuznetsova et al, 2013;Mason and Charniak, 2014), dialogue generation Wu et al, 2018) and style transfer (Lin et al, 2017;. Some editing-based models are proposed to further enhance the retrieved text.…”