Multi-Response Awareness for Retrieval-Based Conversations: Respond with Diversity via Dynamic Representation Learning

Abstract

Conversational systems now attract great attention due to their promising potential and commercial values. To build a conversational system with moderate intelligence is challenging and requires big (conversational) data, as well as interdisciplinary techniques. Thanks to the prosperity of the Web, the massive data available greatly facilitate data-driven methods such as deep learning for human-computer conversational systems. In general, retrieval-based conversational systems apply various matching schema between query utterances and responses, but the classic retrieval paradigm suffers from prominent weakness for conversations: the system finds similar responses given a particular query. For real human-to-human conversations, on the contrary, responses can be greatly different yet all are possibly appropriate. The observation reveals the diversity phenomenon in conversations. In this article, we ascribe the lack of conversational diversity to the reason that the query utterances are statically modeled regardless of candidate responses through traditional methods. To this end, we propose a dynamic representation learning strategy that models the query utterances and different response candidates in an interactive way. To be more specific, we propose a Respond-with-Diversity model augmented by the memory module interacting with both the query utterances and multiple candidate responses. Hence, we obtain dynamic representations for the input queries conditioned on different response candidates. We frame the model as an end-to-end learnable neural network. In the experiments, we demonstrate the effectiveness of the proposed model by achieving a good appropriateness score and much better diversity in retrieval-based conversations between humans and computers.