Multi-Domain Dialogue Acts and Response Co-Generation

Abstract

Generating fluent and informative responses is of critical importance for task-oriented dialogue systems. Existing pipeline approaches generally predict multiple dialogue acts first and use them to assist response generation. There are at least two shortcomings with such approaches. First, the inherent structures of multi-domain dialogue acts are neglected. Second, the semantic associations between acts and responses are not taken into account for response generation. To address these issues, we propose a neural co-generation model that generates dialogue acts and responses concurrently. Unlike those pipeline approaches, our act generation module preserves the semantic structures of multi-domain dialogue acts and our response generation module dynamically attends to different acts as needed. We train the two modules jointly using an uncertainty loss to adjust their task weights adaptively. Extensive experiments are conducted on the largescale MultiWOZ dataset and the results show that our model achieves very favorable improvement over several state-of-the-art models in both automatic and human evaluations.

Hierarchical Transformer for Task Oriented Dialog Systems

“…Using this method they train a domain-aware multi-decoder (DAMD) network for predicting belief, action and response, jointly. As each agent response may cover multiple domains, acts or slots at the same time, Marco (Wang et al, 2020) learns to generate the response by attending over the predicted dialog act sequence at every step of decoding. SimpleTOD (Hosseini-Asl et al, 2020) and SOLOIST (Peng et al, 2020a) are both based on the GPT-2 (Radford et al, 2019) architecture.…”

“…Past research and user-studies have also shown that hierarchy is an important aspect of human conversation (Jurafsky, 2000). But, most previous works based on transformer have focused on training models either as language models (Budzianowski and Vulić, 2019; or as standard (non-hierarchical) Seq2Seq models Zhang et al, 2020a;Wang et al, 2020) with certain task specific extensions. Although arguably, the self-attention mechanism might automatically learn such a scheme during the training process, our empirical results show that forcing this inductive bias by manual design as proposed here leads to better performing models.…”

“…We make a thorough comparison with many recently proposed models in four different experimental settings for dialog response generation task. • We further apply HT-Encoder to a state-of-theart model, Marco (Wang et al, 2020), for taskoriented dialog systems and obtain improved results.…”

Phrase-Level Action Reinforcement Learning for Neural Dialog Response Generation

“…• MarCo (Wang et al, 2020b) extends the idea of HDSA and considers a hierarchical dialog act. The difference is that it co-generates the dialog act sequence and the response jointly.…”

“…In this method, reinforcement learning is performed only on those discrete latent variables, thus the policy optimization is achieved without affecting the language generation. However, LaRL depends on a single vector from the beginning to the end during the response generation, even though a response may often contain more than one dialog act and contents (Wang et al, 2020b). Due to this, a static, global vector tends to be an entangled representation of multiple dialog acts, sentence structure, and contents.…”