Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey

Abstract

Dialogue systems are a popular Natural Language Processing (NLP) task as it is promising in real-life applications. It is also a complicated task since many NLP tasks deserving study are involved. As a result, a multitude of novel works on this task are carried out, and most of them are deep learning based due to the outstanding performance. In this survey, we mainly focus on the deep learning based dialogue systems. We comprehensively review state-of-the-art research outcomes in dialogue systems and analyze them from two angles: model type and system type. Specifically, from the angle of model type, we discuss the principles, characteristics, and applications of different models that are widely used in dialogue systems. This will help researchers acquaint these models and see how they are applied in state-of-the-art frameworks, which is rather helpful when designing a new dialogue system. From the angle of system type, we discuss task-oriented and open-domain dialogue systems as two streams of research, providing insight into the hot topics related. Furthermore, we comprehensively review the evaluation methods and datasets for dialogue systems to pave the way for future research. Finally, some possible research trends are identified based on the recent research outcomes. To the best of our knowledge, this survey is the most comprehensive and up-to-date one at present in the area of dialogue systems and dialogue-related tasks, extensively covering the popular frameworks, topics, and datasets 1 . * Equal contribution ‡ Corresponding author 1 The frameworks, topics, and datasets discussed are originated from the extensive literature review of state-of-the-art research. We have tried our best to cover all but may still omit some works. Readers are welcome to provide suggestions regarding the omissions and mistakes in this article. We also intend to update this article with time as and when new approaches or definitions are proposed and used by the community Preprint. Under review.

Deeper vs Wider: A Revisit of Transformer Configuration

“…Transformer-based language models have achieved promising results on natural language understanding tasks, e.g., Q&A [17,30], relation extraction [29,35] and dialogue system [16]. Recently, on vision tasks, transformers [9,27,28,33] also outperform convolution-based models by a large margin.…”

State-of-the-art in Open-domain Conversational AI: A Survey

“…A systematic survey of recent advances in deep learning-based dialogue systems was conducted by Ni et al (2021), where the authors recognise that dialogue modelling is a complicated task because it involves many related NLP tasks, which are also required to be solved. They categorised dailogue systems by analysing them from two angles: model type and system type (including task-oriented and open-domain conversational systems).…”

State-of-the-Art in Open-Domain Conversational AI: A Survey

“…A systematic survey of recent advances in deep learning-based dialogue systems was conducted by [82], where the authors recognise that dialogue modelling is a complicated task because it involves many related NLP tasks, which are also required to be solved. They categorised dialogue systems by analysing them from two angles: model type and system type (including task-oriented and open-domain conversational systems).…”