Guided Transformer: Leveraging Multiple External Sources for Representation Learning in Conversational Search

Abstract

Asking clarifying questions in response to ambiguous or faceted queries has been recognized as a useful technique for various information retrieval systems, especially conversational search systems with limited bandwidth interfaces. Analyzing and generating clarifying questions have been studied recently but the accurate utilization of user responses to clarifying questions has been relatively less explored. In this paper, we enrich the representations learned by Transformer networks using a novel attention mechanism from external information sources that weights each term in the conversation. We evaluate this Guided Transformer model in a conversational search scenario that includes clarifying questions.In our experiments, we use two separate external sources, including the top retrieved documents and a set of different possible clarifying questions for the query. We implement the proposed representation learning model for two downstream tasks in conversational search; document retrieval and next clarifying question selection. Our experiments use a public dataset for search clarification and demonstrate significant improvements compared to competitive baselines.

Ranking Clarifying Questions Based on Predicted User Engagement

“…However, this method's underlying assumption -the possibility to present a ranked list to the user -does not hold for small-screen interfaces, speech devices, or question-answering systems [21,2]. An alternative approach is to inquire the user about the intent of their query by means of one or more clarifying questions [19,5]. These questions can either have closed form answers, which also have to be generated, or it can be open questions, as done in [7].…”

“…These questions can either have closed form answers, which also have to be generated, or it can be open questions, as done in [7]. Our research's scope is limited to fixed candidate answers, but posing open clarification questions has already been shown to improve search results significantly [5]. For the methodology on generating clarifying questions and their answers, we refer the reader to [19].…”