A Fixed-Point Method for Weighting Terms in Verbose Informational Queries

Abstract

The term weighting and document ranking functions used with informational queries are typically optimized for cases in which queries are short and documents are long. It is reasonable to assume that the presence of a term in a short query reflects some aspect of the topic that is important to the user, and thus rewarding documents that contain the greatest number of distinct query terms is a useful heuristic. Verbose informational queries, such as those that result from cut-and-paste of example text, or that might result from informal spoken interaction, pose a different challenge in which many extraneous (and thus potentially misleading) terms may be present in the query. Modest improvements have been reported from applying supervised methods to learn which terms in a verbose query deserve the greatest emphasis. This paper proposes a novel unsupervised method for weighting terms in verbose informational queries that relies instead on iteratively estimating which terms are most central to the query. The key idea is to use an initial set of retrieval results to define a recursion on the term weight vector that converges to a fixed point representing the vector that optimally describes the initial result set. Experiments with several TREC news and Web test collections indicate that the proposed method often statistically significantly outperforms state of the art supervised methods.

Effective Query Formulation in Conversation Contextualization: A Query Specificity-based Approach

“…Retrieval with verbose queries is also similar to associative document search, a difference being that the verbose queries, in contrast to query-documents, are usually shorter in length, comprised usually of a small number of well-formed sentences [19,31]. IR approaches specifically targeted for verbose queries usually employ a query length normalization component [3,28], or transform the verbose query to a weighted term distribution (assigning higher weights to the terms that better describe the information need) estimated from the top-retrieved documents [31].…”

“…To test the effectiveness of the proposed window-based specificity approach, as a baseline we employ the standard methodology of term extraction from verbose queries [31]. Specifically, in contrast to selecting segments of text (contiguous terms) as potential queries, this baseline method forms the first query by grouping together the most discriminative terms (highest IDFs) and then forms the second query from the next group and so on.…”

“…In contrast to the task of retrieval from verbose queries [19,28,31], which involves returning a single ranked list of documents for a query comprising a large number of words, the task of conversation contextualization, that we address in this paper, involves retrieving relevant information for each potentially 'difficult to comprehend' concept within a conversation, and then associating this information to each such segment of text. In our work, to identify such text segments that are indicative of concepts potentially requiring contextualization, we make use of the specificity estimates from the query performance prediction (QPP) literature.…”

New technique to deal with verbose queries in social book search

“…In this work, Part-of-speech (POS) features of terms are used as observations and the weight levels of the query terms as the hidden states. Drawing on an idea from text summarization, [23] propose an unsupervised method to estimate which term are most central to the query. An initial set of more relevant documents to the original query are used to define a recursion on the query word weight vector that converges to a fixed point representing the vector that optimally describes the initial result set.…”

“…The results increased by 21.8% and 13.4% in terms of MRR and MAP respectively. To tackle the problem of verbosity in natural language queries and improve search engine effectiveness, verbose queries have received more attention in recent years [4,10,23]. Most of them are classified in two main categories: Query Reduction and Query term weighting Approaches.…”