Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation

Abstract

The encoder-decoder dialog model is one of the most prominent methods used to build dialog systems in complex domains. Yet it is limited because it cannot output interpretable actions as in traditional systems, which hinders humans from understanding its generation process. We present an unsupervised discrete sentence representation learning method that can integrate with any existing encoderdecoder dialog models for interpretable response generation. Building upon variational autoencoders (VAEs), we present two novel models, DI-VAE and DI-VST that improve VAEs and can discover interpretable semantics via either auto encoding or context predicting. Our methods have been validated on real-world dialog datasets to discover semantic representations and enhance encoder-decoder models with interpretable generation. 1

Challenges in Building Intelligent Open-domain Dialog Systems

“…Inspired by the early templatebased generation method [26] and statistical machine translation (SMT) [72], sequence-to-sequence (Seq2seq) models [79,85,89,94] have become the most popular choice for dialog generation. Other frameworks, including conditional variational autoencoder (CVAE) [13,33,77,80,124,125] and generative adversarial network (GAN) [40,108], are also applied to dialog generation.…”

Implicit Deep Latent Variable Models for Text Generation

“…Connection to mutual information The proposed latent variable model coincides with (Zhao et al, 2018(Zhao et al, , 2017a where mutual information is introduced into the optimization, based on the following decomposition result (Please see detailed proof in Appendix A):…”

“…Datasets. We consider two mainstream datasets in recent studies (Zhao et al, 2017b(Zhao et al, , 2018Fu et al, 2019;Gu et al, 2018): Switchboard (Godfrey and Holliman, 1997) and Dailydialog (Li et al, 2017c). Switchboard contains 2,400 two-way telephone conversations under 70 specified topics.…”

What You Say and How You Say it: Joint Modeling of Topics and Discourse in Microblog Conversations

“…Though the recent progress in recursive models allows the representation learning from the tree-structured data, previous studies have pointed out that, in practice, sequence models serve as a more simple yet robust alternative (Li et al, 2015). In this work, we follow the common practice in most conversation modeling research (Ritter et al, 2010;Joty et al, 2011;Zhao et al, 2018) to take a conversation as a sequence of turns. To this end, each conversation tree is flattened into root-to-leaf paths.…”