Determining the effectiveness of retrieval algorithms

Frei, Hanspeter; Schäuble, Peter

doi:10.1016/0306-4573(91)90046-o

Cited by 45 publications

(15 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We tested our system using precision, which we found that at the threshold 0.5, the precision is high (97%). We also used the usefulness measurement [7] which had the value of 0.53 indicating that the expanded terms are useful.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Bilingual Search Engine Augmented with Query Expansion

Facundes¹,

Theva-Aksorn²

2007

Seventh International Conference on Intelligent Systems Design and Applications (ISDA 2007)

View full text Add to dashboard Cite

Nowadays, distributed information in the form of linguistic resources is accessible throughout the internet. Search engines have become crucial for information retrieval since users need to find and retrieve relevant information in various languages and forms. The objective of this paper is to investigate the application of query expansion technique to improve cross-language information retrieval in English and Thai. As the method of evaluation of query expansion, we investigate whether the expanded terms are useful for the search. Search CapabilitiesOperator Specification: Operator specification allows a user to logically relate multiple concepts together to define what information is needed. Our system supports queries with various combinations of Boolean operators AND, OR and NOT. Operators are processed from left to right.Query Expansion: The query expansion component employs the dictionary-based technique using part-of-speech information and semantic relations. We use LEXiTRON for this expansion process. When a user submits a query to the system, the query is expanded by adding the terms which are synonyms and related words. However, since a word can have many senses, the appropriate level of expansion must be determined. At the first level expanding, we retain every possible sense, i.e. every possible part-of-speech. In the second level, we perform a part-of-speech disambiguation. If the 1 stlevel expanded words include more than one part-ofspeech, only words that have the same part-of-speech will be selected to the 2 nd -level expanded word. Proper weight calculation is then performed on the resulting words from the 2 nd level. Query Translation:The queries are translated using the same dictionary as in the previous step.Similarity Measurement:The similarity measurement tells how alike two documents are, or how alike a document and a query are. To measure the

show abstract

Section: Discussionmentioning

confidence: 99%

“…Usefulness [7] is an effectiveness measurement which compares two retrieval algorithms; given two algorithms, A and B, u(A,B) expresses whether A is more effective than B or vice versa. A positive value indicates that B is more effective than A; a negative value signifies that B is less effective than A.…”

Section: Usefulness Measurementmentioning

confidence: 99%

Bilingual Search Engine Augmented with Query Expansion

Facundes¹,

Theva-Aksorn²

2007

Seventh International Conference on Intelligent Systems Design and Applications (ISDA 2007)

View full text Add to dashboard Cite

show abstract

“…This is widely accepted in statistics [14], operations research, and other fields. 5 Ranking preference has been used by other researchers for retrieval evaluation as well, for example, in [15].…”

Section: Theoretical Investigationmentioning

confidence: 99%

A geometric framework for data fusion in information retrieval

Crestani

2015

Information Systems

View full text Add to dashboard Cite

“…There have been several attempts to use preference judgments by inferring them from absolute judgments [4] and from click data [8]. Nie et al [9] used preferences for relevance assessments and showed that labelling effort can be reduced by focussing on top ranked documents.…”

Section: Introductionmentioning

confidence: 99%

A document rating system for preference judgements

Bashir

Anderton

et al. 2013

Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

High quality relevance judgments are essential for the evaluation of information retrieval systems. Traditional methods of collecting relevance judgments are based on collecting binary or graded nominal judgments, but such judgments are limited by factors such as inter-assessor disagreement and the arbitrariness of grades. Previous research has shown that it is easier for assessors to make pairwise preference judgments. However, unless the preferences collected are largely transitive, it is not clear how to combine them in order to obtain document relevance scores. Another difficulty is that the number of pairs that need to be assessed is quadratic in the number of documents. In this work, we consider the problem of inferring document relevance scores from pairwise preference judgments by analogy to tournaments using the Elo rating system. We show how to combine a linear number of pairwise preference judgments from multiple assessors to compute relevance scores for every document.

show abstract

Determining the effectiveness of retrieval algorithms

Cited by 45 publications

References 2 publications

Bilingual Search Engine Augmented with Query Expansion

Bilingual Search Engine Augmented with Query Expansion

A geometric framework for data fusion in information retrieval

A document rating system for preference judgements

Contact Info

Product

Resources

About