An End-to-End Goal-Oriented Dialog System with a Generative Natural Language Response Generation

Abstract

Recently advancements in deep learning allowed the development of endto-end trained goal-oriented dialog systems. Although these systems already achieve good performance, some simplifications limit their usage in real-life scenarios.In this work, we address two of these limitations: ignoring positional information and a fixed number of possible response candidates. We propose to use positional encodings in the input to model the word order of the user utterances. Furthermore, by using a feedforward neural network, we are able to generate the output word by word and are no longer restricted to a fixed number of possible response candidates. Using the positional encoding, we were able to achieve better accuracies in the Dialog bAbI Tasks and using the feedforward neural network for generating the response, we were able to save computation time and space consumption.

Multi-task learning to improve natural language understanding

“…There are many appropriate architectures for end-to-end trainable goal-oriented dialog systems [2,4,15] with different approaches for the NLU part; however, what they have in common is that they need a huge amount of training data.…”