Multi-Level Memory for Task Oriented Dialogs

Abstract

Recent end-to-end task oriented dialog systems use memory architectures to incorporate external knowledge in their dialogs. Current work makes simplifying assumptions about the structure of the knowledge base (such as the use of triples to represent knowledge) and combines dialog utterances (context), as well as, knowledge base (KB) results, as part of the same memory. This causes an explosion in the memory size, and makes reasoning over memory, harder. In addition, such a memory design forces hierarchical properties of the data to be fit into a triple structure of memory. This requires the memory reader to learn how to infer relationships across otherwise connected attributes. In this paper we relax the strong assumptions made by existing architectures and use separate memories for modeling dialog context and KB results. Instead of using triples to store KB results, we introduce a novel multilevel memory architecture consisting of cells for each query and their corresponding results. The multi-level memory first addresses queries, followed by results and finally each key-value pair within a result. We conduct detailed experiments on three publicly available task oriented dialog data sets and we find that our method conclusively outperforms current state-of-the-art models. We report a 15-25% increase in both entity F1 and BLEU scores.

Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog

“…Madotto et al (2018) combines end-toend memory network (Sukhbaatar et al, 2015) into sequence generation. Gangi Reddy et al (2019) proposes a multi-level memory architecture which first addresses queries, followed by results and finally each key-value pair within a result. Wu et al (2019a) proposes a global-to-locally pointer mechanism to query the knowledge base.…”

“…Task-oriented dialogue systems (Young et al, 2013) help users to achieve specific goals such as restaurant reservation or navigation inquiry. In recent years, end-to-end methods in the literature usually take the sequence-to-sequence (Seq2Seq) model to generate a response from a dialogue history Madotto et al, 2018;Gangi Reddy et al, 2019;Qin et al, 2019b;Wu et al, 2019a). Taking the dialogue in Figure 1 as an example, to answer the driver's query about the "gas station", the end-to-end dialogue system directly generates system response given the query and a corresponding knowledge base (KB).…”

Unsupervised Learning of KB Queries in Task-Oriented Dialogs

“…SDS (Wen et al, 2017;Williams et al, 2017) use hand-crafted states and state annotations on every utterance in the dialogs-a significant human supervision. End-to-end TOD systems (Reddy et al, 2019;Wu et al, 2019; do not require state annotations but just the KB query annotations. There exist approaches (Chen et al, 2013(Chen et al, , 2015 to induce state annotations in SDS, but we are the first to induce query annotations in end-toend TOD systems.…”

“…An example TOD is shown in Figure 1, where during the conversation (at turn 2), the agent queries the KB based on the user needs, and then suggests the Peking Restaurant based on the retrieved results. Existing end-to-end approaches (Bordes and Weston, 2017;Madotto et al, 2018;Reddy et al, 2019) learn to formulate KB queries using manually annotated queries.…”

UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues

“…In this task, conventional approaches combine Natural Language Understanding (NLU), DST, Dialogue Policy, and NLG, into a pipeline architecture (Wen et al, 2017;Bordes et al, 2016;Liu and Lane, 2017;Liu and Perez, 2017;Williams et al, 2017;Zhao et al, 2017;Jhunjhunwala et al, 2020). Another framework does not explicitly modularize these components but incorporate them through a sequence-to-sequence framework Lei et al, 2018;Yavuz et al, 2019) and a memory-based entity dataset of triplets Madotto et al, 2018;Gangi Reddy et al, 2019;Wu et al, 2019b). These approaches bypass dialogue state and/or act modeling and aim to generate output responses directly.…”