Transferable Dialogue Systems and User Simulators

Abstract

One of the difficulties in training dialogue systems is the lack of training data. We explore the possibility of creating dialogue data through the interaction between a dialogue system and a user simulator. Our goal is to develop a modelling framework that can incorporate new dialogue scenarios through self-play between the two agents. In this framework, we first pre-train the two agents on a collection of source domain dialogues, which equips the agents to converse with each other via natural language. With further fine-tuning on a small amount of target domain data, the agents continue to interact with the aim of improving their behaviors using reinforcement learning with structured reward functions. In experiments on the MultiWOZ dataset, two practical transfer learning problems are investigated: 1) domain adaptation and 2) single-to-multiple domain transfer. We demonstrate that the proposed framework is highly effective in bootstrapping the performance of the two agents in transfer learning. We also show that our method leads to improvements in dialogue system performance on complete datasets.

Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation

“…We calculate the turn accuracy (ACC), joint goal accuracy (JGA) and combined scores (Comb) for these tasks, and compute the average score as the overall metric to measure the model performance. Due to few-shot learning gains increasing attention in various dialog tasks to assess model capability [14,26,90], we conduct experiments under both a full-data setting that uses all training data to fine-tune the models and a few-shot setting that uses only 10% of training data to fine-tune the 3, SPACE-3 outperforms all baselines on all datasets. On the full-data setting and the few-shot setting, SPACE-3 surpasses the PPTOD * with 1.75 and 2.48 absolute average score improvement respectively, indicating the SPACE-3 have better adaptability on all types of dialog tasks.…”

A Unified Dialogue Framework Based on Joint Generation of Dialogue Acts and Responses

“…Wang et al (2020) regarded the dialogue act as a sequence generation task to assist the system in generating replies at each step of the dialogue process. To focus on more diverse dialogue acts, Zhang et al (2020c) devised a data augmentation method that considers multiple dialogue acts to generate system responses simultaneously Tseng et al (2021) designed to first capture the correct dialogue act through the dialogue state, then represent it as a sequence of tokens and generate it through an LSTM. Their strategy for learning dialogue acts can handle multiple different acts during a conversation simultaneously.…”

“…JOUST (Tseng et al, 2021) designed a strategy to obtain the current dialogue act through the dialog state. We denote one of their methods as JOUST+ RL-turn-R. NoisyChannel (Liu et al, 2021) uses the noisy channel model (Yu et al, 2017) to decode dialogue act to generate a higher quality response.…”

Post-processing Networks: Method for Optimizing Pipeline Task-oriented Dialogue Systems using Reinforcement Learning

“…1 Our code is publicly available at https://github. com/nu-dialogue/post-processing-networks However, since each module is processed sequentially, errors in the preceding module can easily propagate to the following ones, and the performance of the entire system cannot be optimized (Tseng et al, 2021). This results in low dialogue performance of the entire system .…”