Linear feature-based models for information retrieval

Metzler, Donald; Croft, W. Bruce

doi:10.1007/s10791-006-9019-z

Cited by 307 publications

(188 citation statements)

References 29 publications

Supporting

Mentioning

188

Contrasting

Order By: Relevance

“…6. COORDINATEASCENT (CA) (Metzler and Croft 2007) is a linear listwise model, where the scores of the query-document pairs are calculated as weighted combinations of the feature values. The weights are tuned by using a coordinate ascent optimization method, where the objective function is an arbitrary evaluation metric given by the user.…”

Section: Methods and Experimental Setupmentioning

confidence: 99%

Tune and mix: learning to rank using ensembles of calibrated multi-class classifiers

et al. 2013

View full text Add to dashboard Cite

In subset ranking, the goal is to learn a ranking function that approximates a gold standard partial ordering of a set of objects (in our case, a set of documents retrieved for the same query). The partial ordering is given by relevance labels representing the relevance of documents with respect to the query on an absolute scale. Our approach consists of three simple steps. First, we train standard multi-class classifiers (AdaBoost.MH and multi-class SVM) to discriminate between the relevance labels. Second, the posteriors of multi-class classifiers are calibrated using probabilistic and regression losses in order to estimate the Bayes-scoring function which optimizes the Normalized Discounted Cumulative Gain (NDCG). In the third step, instead of selecting the best multi-class hyperparameters and the best calibration, we mix all the learned models in a simple ensemble scheme. Mach Learn (2013) 93:261-292 Our extensive experimental study is itself a substantial contribution. We compare most of the existing learning-to-rank techniques on all of the available large-scale benchmark data sets using a standardized implementation of the NDCG score. We show that our approach is competitive with conceptually more complex listwise and pairwise methods, and clearly outperforms them as the data size grows. As a technical contribution, we clarify some of the confusing results related to the ambiguities of the evaluation tools, and propose guidelines for future studies.

show abstract

Section: Methods and Experimental Setupmentioning

confidence: 99%

Tune and mix: learning to rank using ensembles of calibrated multi-class classifiers

et al. 2013

View full text Add to dashboard Cite

show abstract

“…We use this to leverage a semantic kernel function that uses relations between entities as a similarity measure (Section 3.6) and also study an alternative linear kernel. In addition, we use a list-wise learning algorithm, here an implementation from RankLib to directly optimize Mean-Average Precision (MAP) and alternatively Normalized Discounted Cumulative Gain (NDCG), thus addressing the so-called metric divergence problem [30]. We use coordinate ascent as an optimization algorithm, since it has demonstrated good performance on low-dimensional feature spaces with limited training data.…”

Section: Learning To Rank Entities For Web Queriesmentioning

confidence: 99%

Ranking Entities for Web Queries Through Text and Knowledge

Schuhmacher

Dietz

Ponzetto

2015

Proceedings of the 24th ACM International on Conference on Information and Knowledge Management

View full text Add to dashboard Cite

When humans explain complex topics, they naturally talk about involved entities, such as people, locations, or events. In this paper, we aim at automating this process by retrieving and ranking entities that are relevant to understand free-text web-style queries like Argentine British relations, which typically demand a set of heterogeneous entities with no specific target type like, for instance, Falklands_-War or Margaret_Thatcher, as answer. Standard approaches to entity retrieval rely purely on features from the knowledge base. We approach the problem from the opposite direction, namely by analyzing web documents that are found to be query-relevant. Our approach hinges on entity linking technology that identifies entity mentions and links them to a knowledge base like Wikipedia. We use a learning-to-rank approach and study different features that use documents, entity mentions, and knowledge base entities -thus bridging document and entity retrieval. Since established benchmarks for this problem do not exist, we use TREC test collections for document ranking and collect custom relevance judgments for entities. Experiments on TREC Robust04 and TREC Web13/14 data show that: i) single entity features, like the frequency of occurrence within the top-ranked documents, or the query retrieval score against a knowledge base, perform generally well; ii) the best overall performance is achieved when combining different features that relate an entity to the query, its document mentions, and its knowledge base representation.

show abstract

“…Further details can be found in [1]. In our experiments, we consider four popular algorithms across three classes: MART [16] (point-wise), RankBoost [17] (pair-wise), Coordinate Ascent [2] and LambdaMART [18] (list-wise).…”

Section: Related Workmentioning

confidence: 99%

“…Most importantly, these approaches automatically learn the most effective combination of these features in the ranking function based on the available training data. As a result, learning to rank approaches have consistently outperformed the standard bag-of-words retrieval models [2] [3].…”

Section: Introductionmentioning

confidence: 99%

Two-Stage Learning to Rank for Information Retrieval

Dang

Bendersky

Croft

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Current learning to rank approaches commonly focus on learning the best possible ranking function given a small fixed set of documents. This document set is often retrieved from the collection using a simple unsupervised bag-of-words method, e.g. BM25. This can potentially lead to learning a sub-optimal ranking, since many relevant documents may be excluded from the initially retrieved set. In this paper we propose a novel two-stage learning framework to address this problem. We first learn a ranking function over the entire retrieval collection using a limited set of textual features including weighted phrases, proximities and expansion terms. This function is then used to retrieve the best possible subset of documents over which the final model is trained using a larger set of query-and document-dependent features. Empirical evaluation using two web collections unequivocally demonstrates that our proposed two-stage framework, being able to learn its model from more relevant documents, outperforms current learning to rank approaches.

show abstract

Linear feature-based models for information retrieval

Cited by 307 publications

References 29 publications

Tune and mix: learning to rank using ensembles of calibrated multi-class classifiers

Tune and mix: learning to rank using ensembles of calibrated multi-class classifiers

Ranking Entities for Web Queries Through Text and Knowledge

Two-Stage Learning to Rank for Information Retrieval

Contact Info

Product

Resources

About