MOSS: End-to-End Dialog System Framework with Modular Supervision

Abstract

A major bottleneck in training end-to-end task-oriented dialog system is the lack of data. To utilize limited training data more efficiently, we propose Modular Supervision Network (MOSS), an encoder-decoder training framework that could incorporate supervision from various intermediate dialog system modules including natural language understanding, dialog state tracking, dialog policy learning and natural language generation. With only 60% of the training data, MOSS-all (i.e., MOSS with supervision from all four dialog modules) outperforms state-of-the-art models on CamRest676. Moreover, introducing modular supervision has even bigger benefits when the dialog task has a more complex dialog state and action space. With only 40% of the training data, MOSS-all outperforms the state-of-the-art model on a complex laptop network trouble shooting dataset, LaptopNetwork, that we introduced. LaptopNetwork consists of conversations between real customers and customer service agents in Chinese. Moreover, MOSS framework can accommodate dialogs that have supervision from different dialog modules at both framework level and model level. Therefore, MOSS is extremely flexible to update in real-world deployment.

GraghVQA: Language-Guided Graph Neural Networks for Graph-based Visual Question Answering

“…One major difference between our Graph Reasoning Module and standard GNN is that, we want the message passing in layer L conditioned on the L th instruction vector. Inspired by language model type condition (Liang et al, 2020b), we adopt a general design that is compatible with any graph neural network design: Before running the L th GNN layer, we concatenate the L th instruction vector to every node and edge feature from the previous layer. Specifically,…”

A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning

“…We borrow some elements from the Sequicity ) model, such as representing the belief state as a natural language sequence (a text span), and using copy-augmented Seq2Seq learning (Gu et al, 2016). But compared to Sequicity and all its follow-up works Shu et al, 2019;Liang et al, 2020), a feature in our LABES-S2S model is that the transition between belief states across turns and the dependency between system responses and belief states are well statistically modeled. This new design results in a completely different graphical model structure, which enables rigorous probabilistic variational learning.…”

“…E2E Models: E2E models can be divided into three sub-categories. The TSCP , SEDST , FSDM (Shu et al, 2019), MOSS (Liang et al, 2020) and DAMD are based on the copy-augmented Seq2Seq learning framework proposed by . LIDM (Wen et al, 2017a), SFN (Mehri et al, 2019) and UniConv (Le et al, 2020a) are modular designed, connected through neural states and trained end-to-end.…”

“…The system response reveals the belief state either directly in the form of word repetition (red), or indirectly in the form of the database query result (green) determined by the belief state. or within end-to-end (E2E) trainable dialog systems (Wen et al, 2017a,b;Liu and Lane, 2017;Shu et al, 2019;Liang et al, 2020;.…”

End-to-end ASR to jointly predict transcriptions and linguistic annotations

“…1(b) shows an O2O model with a conditional chain mapping. This method for multiple sequence modeling has been applied to dialog modeling (Liang et al, 2020), speaker diarization (Fujita et al, 2020a), and multi-speaker ASR (Shi et al, 2020). Unlike the O2M model, this model can predict a variable number of output sequences while explicitly considering dependencies between the multiple sequences based on the probabilistic chain rule.…”