Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems

Madotto, Andrea; Wu, Chien-Sheng; Fung, Pascale

doi:10.18653/v1/p18-1136

Cited by 252 publications

(279 citation statements)

References 26 publications

Supporting

Mentioning

277

Contrasting

Order By: Relevance

“…The Mem2Seq (Madotto et al, 2018) incorporates structured knowledge into the end-to-end task-oriented dialogue. introduces factmatching and knowledge-diffusion to generate meaningful, diverse and natural responses using structured knowledge triplets.…”

Section: Introductionmentioning

confidence: 99%

Incremental Transformer with Deliberation Decoder for Document Grounded Conversations

Li¹,

Niu²,

Meng³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Document Grounded Conversations is a task to generate dialogue responses when chatting about the content of a given document. Obviously, document knowledge plays a critical role in Document Grounded Conversations, while existing dialogue models do not exploit this kind of knowledge effectively enough. In this paper, we propose a novel Transformerbased architecture for multi-turn document grounded conversations. In particular, we devise an Incremental Transformer to encode multi-turn utterances along with knowledge in related documents. Motivated by the human cognitive process, we design a two-pass decoder (Deliberation Decoder) to improve context coherence and knowledge correctness. Our empirical study on a real-world Document Grounded Dataset proves that responses generated by our model significantly outperform competitive baselines on both context coherence and knowledge relevance. * * Fandong Meng is the corresponding author of the paper. This work was done when Zekang Li was interning at Pattern Recognition Center, WeChat AI, Tencent.

show abstract

Section: Introductionmentioning

confidence: 99%

Incremental Transformer with Deliberation Decoder for Document Grounded Conversations

Li¹,

Niu²,

Meng³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…The hyper-parameter settings are adopted as the best practice settings for each training set following the Madotto's (Madotto et al, 2018) and best experimental results on baselines SEQ2SEQ and Mem2Seq. Detailed models and their settings are as follows:…”

Section: Baselines and Training Setupmentioning

confidence: 99%

“…Shang et al, 2015), show that training a fully data-driven end-to-end model is a promising way to build domain-agnostic dialogue system. Their models mostly try to use the attention mechanism, including memory networks techniques, to fetch the most similar knowledge (Sukhbaatar et al, 2015), then incorporate grounding knowledge into a seq2seq neural model to generate a suitable re-sponse (Madotto et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

Task-Oriented Conversation Generation Using Heterogeneous Memory Networks

Lin

Huang

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

How to incorporate external knowledge into a neural dialogue model is critically important for dialogue systems to behave like real humans. To handle this problem, memory networks are usually a great choice and a promising way. However, existing memory networks do not perform well when leveraging heterogeneous information from different sources. In this paper, we propose a novel and versatile external memory networks called Heterogeneous Memory Networks (HMNs), to simultaneously utilize user utterances, dialogue history and background knowledge tuples. In our method, historical sequential dialogues are encoded and stored into the context-aware memory enhanced by gating mechanism while grounding knowledge tuples are encoded and stored into the context-free memory. During decoding, the decoder augmented with HMNs recurrently selects each word in one response utterance from these two memories and a general vocabulary. Experimental results on multiple real-world datasets show that HMNs significantly outperform the state-of-the-art datadriven task-oriented dialogue models in most domains.

show abstract

“…Moreover, they didn't utilize user intent during modeling. In [16], the authors used a memory-to-sequence model that uses multi-hop attention over memories to help in learning correlations between memories which results in faster trained model with a stable performance. As for using joint learning to support end-to-end dialogue agent the work introduced by [11] showed state-of-the-art results where they used an attention based RNN for the joint learning of intent detection and slot filling.…”

Section: Related Workmentioning

confidence: 99%

Incorporating Joint Embeddings into Goal-Oriented Dialogues with Multi-task Learning

2019

View full text Add to dashboard Cite

Attention-based encoder-decoder neural network models have recently shown promising results in goal-oriented dialogue systems. However, these models struggle to reason over and incorporate state-full knowledge while preserving their end-to-end text generation functionality. Since such models can greatly benefit from user intent and knowledge graph integration, in this paper we propose an RNN-based end-to-end encoder-decoder architecture which is trained with joint embeddings of the knowledge graph and the corpus as input. The model provides an additional integration of user intent along with text generation, trained with multi-task learning paradigm along with an additional regularization technique to penalize generating the wrong entity as output. The model further incorporates a Knowledge Graph entity lookup during inference to guarantee the generated output is state-full based on the local knowledge graph provided. We finally evaluated the model using the BLEU score, empirical evaluation depicts that our proposed architecture can aid in the betterment of task-oriented dialogue system's performance.

show abstract

Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems

Cited by 252 publications

References 26 publications

Incremental Transformer with Deliberation Decoder for Document Grounded Conversations

Incremental Transformer with Deliberation Decoder for Document Grounded Conversations

Task-Oriented Conversation Generation Using Heterogeneous Memory Networks

Incorporating Joint Embeddings into Goal-Oriented Dialogues with Multi-task Learning

Contact Info

Product

Resources

About