Deep Learning for Dialogue Systems

Abstract

In the past decade, goal-oriented spoken dialogue systems have been the most prominent component in today's virtual personal assistants. The classic dialogue systems have rather complex and/or modular pipelines. The advance of deep learning technologies has recently risen the applications of neural models to dialogue modeling. However, how to successfully apply deep learning based approaches to a dialogue system is still challenging. Hence, this tutorial is designed to focus on an overview of the dialogue system development while describing most recent research for building dialogue systems and summarizing the challenges, in order to allow researchers to study the potential improvements of the state-of-the-art dialogue systems. The tutorial material is available at http://deepdialogue. miulab.tw. Tutorial OverviewWith the rising trend of artificial intelligence, more and more devices have incorporated goal-oriented spoken dialogue systems. Among popular virtual personal assistants, Microsoft's Cortana, Apple's Siri, Amazon Alexa, Google Assistant, and Facebook's M, have incorporated dialogue system modules in various devices, which allow users to speak naturally in order to finish tasks more efficiently.The traditional conversational systems have rather complex and/or modular pipelines. The advance of deep learning technologies has recently risen the applications of neural models to dialogue modeling. Nevertheless, applying deep learning technologies for building robust and scalable dialogue systems is still a challenging task and an open research area as it requires deeper understanding of the classic pipelines as well as detailed knowledge on the benchmark of the models of the prior work and the recent state-of-the-art work.The goal of this tutorial is to provide the audience with developing trend of the dialogue systems, and a roadmap to get them started with the related work. In the first section of the tutorial, we motivate the work on conversation-based intelligent agents, in which the core underlying system is task-oriented dialogue systems. The second and third sections describe different approaches using deep learning for each component in the dialogue system and how it is evaluated. The last two sections focus on discussing the recent trends and current challenges on dialogue system technology and summarize the challenges and conclusions. Then the detailed content is described as follows.2 Outline

Zero-Shot Adaptive Transfer for Conversational Language Understanding

“…A number of deep learning approaches have been applied to the problem of language understanding in recent years (Deng et al 2012;Mesnil et al 2015;Celikyilmaz, Deng, and Hakkani-Tür 2018). For a thorough overview of deep learning methods in conversational language understanding, we refer the readers to (Chen, Celikyilmaz, and Hakkani-Tür 2017).…”

Neural Approaches to Conversational AI

“…(Yih et al, 2015(Yih et al, , 2016Gao, 2017) reviewed deep learning approaches to a wide range of NLP and IR tasks, including dialogue. (Chen et al, 2017b) is a recent tutorial on dialogue mainly focusing on task-oriented agents. (Serban et al, 2015) gave a good survey of public dialogue datasets that can used to develop dialogue agents.…”

Building a Legal Dialogue System: Development Process, Challenges and Opportunities

“…By contrast, retrieval-based systems map an input to a response from a repository of responses. A recent survey on dialog systems can be found in [3,4,7].…”