Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems

Abstract

Intelligent personal assistant systems with either text-based or voice-based conversational interfaces are becoming increasingly popular around the world. Retrieval-based conversation models have the advantages of returning fluent and informative responses. Most existing studies in this area are on open domain "chit-chat" conversations or task / transaction oriented conversations. More research is needed for information-seeking conversations. There is also a lack of modeling external knowledge beyond the dialog utterances among current conversational models. In this paper, we propose a learning framework on the top of deep neural matching networks that leverages external knowledge for response ranking in information-seeking conversation systems. We incorporate external knowledge into deep neural models with pseudo-relevance feedback and QA correspondence knowledge distillation. Extensive experiments with three information-seeking conversation data sets including both open benchmarks and commercial data show that, our methods outperform various baseline methods including several deep text matching models and the state-of-the-art method on response selection in multi-turn conversations. We also perform analysis over different response types, model variations and ranking examples. Our models and research findings provide new insights on how to utilize external knowledge with deep neural models for response selection and have implications for the design of the next generation of information-seeking conversation systems.

BERT with History Answer Embedding for Conversational Question Answering

“…In addition to ConvQA, there are other related works focused on conversational search. For example, neural approaches are widely adopted to train a model to ask questions proactively [19], predict user intent [11], predict next question [17], and incorporate external knowledge in response ranking [18]. In addition, several observational studies are also conducted [10,15] to inform the design of conversational search systems.…”

Towards a model for spoken conversational search

“…In recent years, interest in SCS has grown, as speech technology [56] and machine learning for spoken systems [57] have developed. A range of SDS are available, from question answering to semi-conversational systems [28].…”

A Deep Look into neural ranking models for information retrieval

“…Xu et al[137] designed a Recall gate, where domain knowledge can be transformed into the extra global memory of LSTM, with the aim of enhancing LSTM by cooperating with its local memory to capture the implicit semantic relevance between sentences within conversations.Beyond structured knowledge in knowledge bases, other research has explored how to integrate external knowledge from unstructured texts, which are more common for information on the Web. Yang et al[30] studied response ranking in information-seeking conversations and proposed two effective meth-ods to incorporate external knowledge into neural ranking models with pseudorelevance feedback (PRF) and QA correspondence knowledge distillation. They proposed to extract the "correspondence" regularities between question and answer terms from retrieved external QA pairs as external knowledge to help response selection.…”