Few-Shot Conversational Dense Retrieval

Abstract

Dense retrieval (DR) has the potential to resolve the query understanding challenge in conversational search by matching in the learned embedding space. However, this adaptation is challenging due to DR models' extra needs for supervision signals and the longtail nature of conversational search. In this paper, we present a Conversational Dense Retrieval system, ConvDR, that learns contextualized embeddings for multi-turn conversational queries and retrieves documents solely using embedding dot products. In addition, we grant ConvDR few-shot ability using a teacher-student framework, where we employ an ad hoc dense retriever as the teacher, inherit its document encodings, and learn a student query encoder to mimic the teacher embeddings on oracle reformulated queries. Our experiments on TREC CAsT and OR-QuAC demonstrate ConvDR's effectiveness in both few-shot and fully-supervised settings. It outperforms previous systems that operate in the sparse word space, matches the retrieval accuracy of oracle query reformulations, and is also more efficient thanks to its simplicity. Our analyses reveal that the advantages of ConvDR come from its ability to capture informative context while ignoring the unrelated context in previous conversation rounds. This makes ConvDR more effective as conversations evolve while previous systems may get confused by the increased noise from previous turns. Our code is publicly available at https://github.com/thunlp/ConvDR.

An animated picture says at least a thousand words: Selecting Gif-based Replies in Multimodal Dialog

“…We follow the evaluation setup from other retrieval-based dialog systems (e.g. Yu et al, 2021;Kumar and Callan, 2020) and use normalized Discounted Cumulative Gain (nDCG), which measures whether more appropriate gif responses are ranked higher. A gif's appropriateness score is the sum of annotators' ratings.…”

Conversational Information Seeking

“…In these representations the query and history is combined into one or more vectors issued as queries to a dense retrieval system. Yu et al (2021) encoded the history representation with a dense vector that is learned with a teacher-student model to mimic a dense representation of the manually rewritten query. The model for multiple turns uses composition with dense retrieval approaches similar to those in multi-hop QA (Khattab et al, 2021), but applied to a conversational context.…”

“…Instead of the multi-stage cascade architecture, an alternative is end-to-end approaches based upon dense retrieval, sometimes referred to as Conversational Dense Retrieval (ConvDR) (Yu et al, 2021). The distinguishing feature is that retrieval and conversation are encoded with a dense vector rather than an explicit word-based query.…”

Neural Approaches to Conversational AI

“…Next, we will describe a method of training ConvDR models in a few-shot learning setting. Yu et al [2021a] propose a few-shot learning method to train a ConvDR model using the teacherstudent knowledge distillation framework [Hinton et al, 2015], where the query encoder of ConvDR (student) learns to produce the output of the query encoder of a well-trained ad hoc dense retrieval model (teacher). Specifically, the method assumes that we have 1.…”