Making Information Seeking Easier: An Improved Pipeline for Conversational Search

Abstract

This paper presents a highly effective pipeline for passage retrieval in a conversational search setting. The pipeline comprises of two components: Conversational Term Selection (CTS) and Multi-View Reranking (MVR). CTS is responsible for performing the first-stage of passage retrieval. Given an input question, it uses a BERT-based classifier (trained with weak supervision) to de-contextualize the input by selecting relevant terms from the dialog history. Using the question and the selected terms, it issues a query to a search engine to perform the first-stage of passage retrieval. On the other hand, MVR is responsible for contextualized passage reranking. It first constructs multiple views of the information need embedded within an input question. The views are based on the dialog history and the top documents obtained in the first-stage of retrieval. It then uses each view to rerank passages using BERT (fine-tuned for passage ranking). Finally, MVR performs a fusion over the rankings produced by the individual views. Experiments show that the above combination improves first-state retrieval as well as the overall accuracy in a reranking pipeline. On the key metric of NDCG@3, the proposed combination achieves a relative performance improvement of 14.8% over the state-of-the-art baseline and is also able to surpass the Oracle.

Contextualized Query Embeddings for Conversational Search

“…CQR + BM25 + BERT-base: latency = 5,350 ms QuReTec (Voskarides et al, 2020) .476 Few-Shot Rewriter (Yu et al, 2020) .492 3CQR + BM25 + BERT-base: latency = 8,025 ms (est.) MVR (Kumar and Callan, 2020) .565…”

“…(2) There is limited data regarding conversational search for model training. To address the aforementioned challenges, existing papers (Lin et al, 2021c;Yu et al, 2020;Voskarides et al, 2020;Kumar and Callan, 2020) take a multi-stage pipeline approach. They train a conversational query reformulation (CQR) model using publicly available datasets (Elgohary et al, 2019;Quan et al, 2019) and feed the automatically decontextualized queries to an off-the-shelf IR pipeline (Nogueira and Cho, 2019).…”

“…For example, we can reuse the representative ad hoc retrieval pipeline comprised of BM25 + BERT re-ranking for F ir φ , then conduct the CQR task for F CQR ϕ . Specifically, the most common current approach (Lin et al, 2021c;Voskarides et al, 2020;Vakulenko et al, 2020;Kumar and Callan, 2020;Yu et al, 2020) is to fine-tune a pretrained language model (LM) supervised by decontextualized queries manually rewritten by humans, and then use the fine-tuned LM to reformulate user queries for BM25 retrieval and BERT re-ranking, as illustrated in Figure 1(a). While effective, this approach has two limitations: (1) although mimicking the way humans rewrite queries is reasonable, it hypothesizes that the optimal decontextualized queries are manually rewritten queries, which may not be true; (2) the CQR and IR modules rely on computation-demanding pretrained LMs; thus, when combined together, they are often too slow for practical applications.…”

Few-Shot Conversational Dense Retrieval

“…These solutions aim to reformulate the conversational query to an ad hoc query in the sparse bag-of-words space and then leverage standard sparse retrieval pipelines such as BM25 and BERT ranker [8,22,29]. By design, the vocabulary mismatch problem [6] in these conversational search systems is more severe than their corresponding ad hoc search systems, as the extra query reformulation step is an additional source of errors.…”

Conversational Entity Linking: Problem Definition and Datasets

“…Entity linking in conversations. Research on conversational entity linking has been mainly focused on employing traditional entity linking and named entity recognition methods in conversational and QA systems [7,14,15,38,39,56]. Entity linking is also used in multi-party conversations to connect mentions across different parts of dialogues and mapping to their corresponding character [15].…”