Predicting query performance

Cronen-Townsend, Steve; Zhou, Yun; Croft, W. Bruce

doi:10.1145/564426.564429

Cited by 110 publications

(120 citation statements)

References 0 publications

Supporting

Mentioning

118

Contrasting

Unclassified

Order By: Relevance

“…To find indicators or features to represent the quality of each query (record), we draw on the large body of previous work on query quality prediction [23,24,25]. These include predicting the quality of queries using either pre-retrieval indicators like Query Scope, that are calculated for a query as a whole, or post-retrieval indicators like Query Clarity, that involve assessing the results of an initial retrieval and hence are more expensive to compute.…”

Section: Quality Factorsmentioning

confidence: 99%

See 1 more Smart Citation

Literature Consistency of Bioinformatics Sequence Databases is Effective for Assessing Record Quality

Bouadjenek

Verspoor

Zobel

2017

Preprint

View full text Add to dashboard Cite

Bioinformatics sequence databases such as Genbank or UniProt contain hundreds of millions of records of genomic data. These records are derived from direct submissions from individual laboratories, as well as from bulk submissions from large-scale sequencing centres; their diversity and scale means that they suffer from a range of data quality issues including errors, discrepancies, redundancies, ambiguities, incompleteness, and inconsistencies with the published literature. In this work, we seek to investigate and analyze the data quality of sequence databases from the perspective of a curator, who must detect anomalous and suspicious records.Specifically, we emphasize the detection of inconsistent records with respect to the literature.Focusing on GenBank, we propose a set of 24 quality indicators, which are based on treating a record as a query into the published literature, and then use query quality predictors. We then carry out an analysis that shows that the proposed quality indicators and the quality of the records have a mutual relationship, in which one depends on the other. We propose to represent recordliterature consistency as a vector of these quality indicators. By reducing the dimensionality of this representation for visualization purposes using Principal Component Analysis, we show that records which have been reported as inconsistent with the literature fall roughly in the same area, and therefore share similar characteristics. By manually analyzing records not previously known to be erroneous that fall in the same area than records know to be inconsistent, we show that 1 record out of 4 is inconsistent with respect to the literature. This high density of inconsistent record opens the way towards the development of automatic methods for the detection of faulty records. We conclude that literature inconsistency is a meaningful strategy for identifying suspicious records.

show abstract

Section: Quality Factorsmentioning

confidence: 99%

“…The clarity score is post-retrieval query quality factor, which has been developed by Cronen-Townsend et al [23]. It is simply the Kullback-Leibler divergence of the query model from the collection model.…”

Section: Clarity Scorementioning

confidence: 99%

Literature Consistency of Bioinformatics Sequence Databases is Effective for Assessing Record Quality

Bouadjenek

Verspoor

Zobel

2017

Preprint

View full text Add to dashboard Cite

show abstract

“…The simplified query clarity measure involves the simplification of the query model, which is estimated using the frequency of the query term within the query as given by qtf l , where qtf refers to the frequency of a query term within the query and l refers to query length. The average idf score, which has previously been used to predict query performance (Cronen-Townsend, Zhou, & Croft, 2002;de Loupy & Bellot, 2000;Kwok, 2005) is defined as as avg idf idf q l…”

Section: Interacting Factorsmentioning

confidence: 99%

“…We use the Spearman rank correlation as it is a nonparametric measure of correlation. It has also been used in past literature on query performance prediction (Cronen-Townsend et al, 2002;He & Ounis, 2006). The Spearman rank correlation is given by ρ = Table 7, we present the Spearman rank correlation of various query features with query performance measures.…”

Section: Feature Evaluationmentioning

confidence: 99%

Query‐performance prediction for effective query routing in domain‐specific repositories

Sarnikar

Zhang²,

Zhao

2014

Asso for Info Science & Tech

View full text Add to dashboard Cite

The effective use of corporate memory is becoming increasingly important because every aspect of e-business requires access to information repositories. Unfortunately, less-than-satisfying effectiveness in state-of-the-art information-retrieval techniques is well known, even for some of the best search engines such as Google. In this study, the authors resolve this retrieval ineffectiveness problem by developing a new framework for predicting query performance, which is the first step toward better retrieval effectiveness. Specifically, they examine the relationship between query performance and query context. A query context consists of the query itself, the document collection, and the interaction between the two. The authors first analyze the characteristics of query context and develop various features for predicting query performance. Then, they propose a context-sensitive model for predicting query performance based on the characteristics of the query and the document collection. Finally, they validate this model with respect to five real-world collections of documents and demonstrate its utility in routing queries to the correct repository with high accuracy.

show abstract

“…The very short queries common in the web search environment represent a challenge for existing retrieval and ranking methodologies, because query length is known to be positively related to effectiveness of retrieval results (Belkin et al, 2003). Cronen-Townsend, Zhou, and Croft (2002) developed a method to predict query performance by using relative entropy between query and collection language models. The similar query prediction algorithm applied on GOV2 (web collection) shows that it is difficult to predict query performance in a large web collection (Carmel, Yom-Tov, Darlow, & Pelleg, 2006).…”

Section: Information Need Query Formulation and Rankingmentioning

confidence: 99%

Real‐time user interest modeling for real‐time ranking

Liu

Turtle

2013

J Am Soc Inf Sci Tec

View full text Add to dashboard Cite

User interest as a very dynamic information need is often ignored in most existing information retrieval systems. In this research, we present the results of experiments designed to evaluate the performance of a real‐time interest model (RIM) that attempts to identify the dynamic and changing query level interests regarding social media outputs. Unlike most existing ranking methods, our ranking approach targets calculation of the probability that user interest in the content of the document is subject to very dynamic user interest change. We describe 2 formulations of the model (real‐time interest vector space and real‐time interest language model) stemming from classical relevance ranking methods and develop a novel methodology for evaluating the performance of RIM using Amazon Mechanical Turk to collect (interest‐based) relevance judgments on a daily basis. Our results show that the model usually, although not always, performs better than baseline results obtained from commercial web search engines. We identify factors that affect RIM performance and outline plans for future research.

show abstract

Predicting query performance

Cited by 110 publications

References 0 publications

Literature Consistency of Bioinformatics Sequence Databases is Effective for Assessing Record Quality

Literature Consistency of Bioinformatics Sequence Databases is Effective for Assessing Record Quality

Query‐performance prediction for effective query routing in domain‐specific repositories

Real‐time user interest modeling for real‐time ranking

Contact Info

Product

Resources

About