Toward Voice Query Clarification

Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

Crestani

et al. 2019

221

290

Users often fail to formulate their complex information needs in a single query. As a consequence, they may need to scan multiple result pages or reformulate their queries, which may be a frustrating experience. Alternatively, systems can improve user satisfaction by proactively asking questions of the users to clarify their information needs. Asking clarifying questions is especially important in conversational systems since they can only return a limited number of (often only one) result(s).In this paper, we formulate the task of asking clarifying questions in open-domain information-seeking conversational systems. To this end, we propose an offline evaluation methodology for the task and collect a dataset, called Qulac, through crowdsourcing. Our dataset is built on top of the TREC Web Track 2009-2012 data and consists of over 10K question-answer pairs for 198 TREC topics with 762 facets. Our experiments on an oracle model demonstrate that asking only one good question leads to over 170% retrieval performance improvement in terms of P@1, which clearly demonstrates the potential impact of the task. We further propose a retrieval framework consisting of three components: question retrieval, question selection, and document retrieval. In particular, our question selection model takes into account the original query and previous question-answer interactions while selecting the next question. Our model significantly outperforms competitive baselines. To foster research in this area, we have made Qulac publicly available.

mentioning

confidence: 99%

Asking Clarifying Questions in Open-Domain Information-Seeking Conversations

Aliannejadi

Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

Crestani

et al. 2019

221

290

“…More recently, with the developments around machine learning and natural language processing (NLP), the search interactions in which both input and output are presented via speech has received considerable attention [33,58]. This more natural way of expressing an information need via speech is considered an advantage of the spoken conversational search interaction paradigm.…”

Section: Uni-modal Information Seekingmentioning

confidence: 99%

“…Recent conversational approaches to finding information through dialogue have mainly been viewed from natural language interactions (i.e., search in which input and output are mediated via "conversations" [33,51,60]). Even though several mentions towards conversational multi-modal systems suggested an expansion in the in-and output options [28,46], limited research has been devoted to interacting with IS systems in a range of modalities which complement each other.…”

Section: Mmcis: Definitions and Applicationsmentioning

confidence: 99%

Towards Multi-Modal Conversational Information Seeking

Deldjoo

Trippas

Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

2021

Recent research on conversational information seeking (CIS) mostly focuses on uni-modal interactions and information items. This perspective paper highlights the importance of moving towards developing and evaluating multi-modal conversational information seeking (MMCIS) systems as they enable us to leverage richer context, overcome errors, and increase accessibility. We bridge the gap between the multi-modal and CIS research and provide a formal definition for MMCIS. We discuss potential opportunities and research challenges in designing, implementing, and evaluating MMCIS systems. Based on this research, we propose and implement a practical open-source framework for facilitating MMCIS research.

“…Recently, it attracted much attention in the information retrieval literature [1,14,22,29,30]. For instance, Kiesel et al [16] investigated the impact of voice query clarification on user satisfaction. Their study showed that users like to be prompted for clarification.…”

Section: Related Workmentioning

confidence: 99%

Mimics

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

Lueck

Chen

et al. 2020

Search clarification has recently attracted much attention due to its applications in search engines. It has also been recognized as a major component in conversational information seeking systems. Despite its importance, the research community still feels the lack of a large-scale dataset for studying different aspects of search clarification. In this paper, we introduce MIMICS, a collection of search clarification datasets for real web search queries sampled from the Bing query logs. Each clarification in MIMICS is generated by a Bing production algorithm and consists of a clarifying question and up to five candidate answers. MIMICS contains three datasets: (1) MIMICS-Click includes over 400k unique queries, their associated clarification panes, and the corresponding aggregated user interaction signals (i.e., clicks). (2) MIMICS-ClickExplore is an exploration data that includes aggregated user interaction signals for over 60k unique queries, each with multiple clarification panes. (3) MIMICS-Manual includes over 2k unique real search queries. Each query-clarification pair in this dataset has been manually labeled by at least three trained annotators. It contains graded quality labels for the clarifying question, the candidate answer set, and the landing result page for each candidate answer. MIMICS is publicly available for research purposes, 1 thus enables researchers to study a number of tasks related to search clarification, including clarification generation and selection, user engagement prediction for clarification, click models for clarification, and analyzing user interactions with search clarification. We also release the results returned by the Bing's web search API for all the queries in MIMICS. This would allow researchers to utilize search results for the tasks related to search clarification.