Relationship between index term specificity and relevance judgment

Kim, Giyeong

doi:10.1016/j.ipm.2005.12.004

Cited by 14 publications

(11 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Many studies on various aspects of human information behavior are related to relevance behavior, but are not included here for space reasons. Examples include studies on decisions about documents in reading and citing (Wang & White, 1999), on judgment of cognitive authority and information quality (Rieh & Belkin, 2000), on users' assessments of Web pages (Tombros, Ruthven, & Jose, 2005), or on relation between search terms, index terms, and documents judged as to relevance (Kim, 2006). Kelly (2005) reviewed a host of studies about human decisions during interaction with the Web (or other information resources); the focus was on decisions as to what to examine, retain (save, print), reference, annotate, and the like.…”

Section: Prologue To Part Iii: How It Is Connected and What This Workmentioning

confidence: 99%

Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance

Saračević

2007

J. Am. Soc. Inf. Sci.

209

214

View full text Add to dashboard Cite

Relevance is a, if not even the, key notion in information science in general and information retrieval in particular. This two-part critical review traces and synthesizes the scholarship on relevance over the past 30 years or so and provides an updated framework within which the still widely dissonant ideas and works about relevance might be interpreted and related. It is a continuation and update of a similar review that appeared in 1975 under the same title, considered here as being Part I. The present review is organized in two parts: Part II addresses the questions related to nature and manifestations of relevance, and Part III addresses questions related to relevance behavior and effects. In Part II, the nature of relevance is discussed in terms of meaning ascribed to relevance, theories used or proposed, and models that have been developed. The manifestations of relevance are classified as to several kinds of relevance that form an interdependent system of relevancies. In Part III, relevance behavior and effects are synthesized using experimental and observational works that incorporated data. In both parts, each section concludes with a summary that in effect provides an interpretation and synthesis of contemporary thinking on the topic treated or suggests hypotheses for future research. Analyses of some of the major trends that shape relevance work are offered in conclusions. Prologue to Part III: How It Is Connected and What This Work Is All AboutTo provide a continuation from the preceding Part II, a few basic descriptions about this work are repeated.As stated in the Preface to Part II, in 1975 I published a review about relevance under the same title, without, of course, "Part I" in the title (Saracevic, 1975). There was no plan then to have another related review 30 years later-but things happen. The intent of the 1975 work was "to explore the meaning of relevance as it has evolved in information science and to provide a framework within which various interpretations of relevance can be related" (Saracevic, 1975, p. 321).Building on the examination of relevance in the preceding (1975) review, this work (2007) follows the travails of relevance in information science for the past 30 years. It is an update. The aim of this work is still substantially the same: It is an attempt to trace the evolution of thinking on relevance in information science for the past three decades and to provide an updated, contemporary framework within which the still widely dissonant ideas on relevance might be interpreted and related to one another.The organization of the present review, offered in two parts, addresses the questions related to relevance nature, manifestations, behavior, and effects. Following the Introduction and a Historical Footnote, the preceding Part II (this issue, pp. 1915-1933) started with a general section on the nature of relevance by synthesizing its meanings, following with sections on theories and models of relevance that are, in effect, further elaborations on the nature of relevance. Part ...

show abstract

Section: Prologue To Part Iii: How It Is Connected and What This Workmentioning

confidence: 99%

Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance

Saračević

2007

J. Am. Soc. Inf. Sci.

209

214

View full text Add to dashboard Cite

show abstract

“…More specifically, we investigate three facets described here: Posting specificity PSpe ( Q ): expresses the uniqueness of query words in the index collection; the basic assumption behind posting specificity is that the fewer documents involved by query words, the more specific the query topics are (Kim, ). Index specificity ISpe ( Q ): highlights the variation in information amount that query words carry (Pirkola & Jarvelin, ). Intuitively, we believe that the larger the index variation is, the less specific the query topics are. Hierarchical specificity HSpe ( Q ): it is based on the query terms' depth of meaning defined in a reference terminology through the “is‐a” taxonomic relation (Kim, ). The basic underlying assumption is that the more specific concepts are involved by the query words, the more specific the query topics are.…”

Section: Methodsmentioning

confidence: 99%

“…More specifically, we investigate three facets described here: 1. Posting specificity PSpe(Q): expresses the uniqueness of query words in the index collection; the basic assumption behind posting specificity is that the fewer documents involved by query words, the more specific the query topics are (Kim, 2006). 2.…”

Section: Data Test Collections Text Retrieval Conference (Trec)mentioning

confidence: 99%

Analysis of biomedical and health queries: Lessons learned fromTRECandCLEFevaluation benchmarks

Tamine

Chouquet

Palmer

2015

Asso for Info Science & Tech

View full text Add to dashboard Cite

A large body of research work examined, from both the query side and the user behavior side, the characteristics of medical‐ and health‐related searches. One of the core issues in medical information retrieval (IR) is diversity of tasks that lead to diversity of categories of information needs and queries. From the evaluation perspective, another related and challenging issue is the limited availability of appropriate test collections allowing the experimental validation of medically task oriented IR techniques and systems. In this paper, we explore the peculiarities of TREC and CLEF medically oriented tasks and queries through the analysis of the differences and the similarities between queries across tasks, with respect to length, specificity, and clarity features and then study their effect on retrieval performance. We show that, even for expert oriented queries, language specificity level varies significantly across tasks as well as search difficulty. Additional findings highlight that query clarity factors are task dependent and that query terms specificity based on domain‐specific terminology resources is not significantly linked to term rareness in the document collection. The lessons learned from our study could serve as starting points for the design of future task‐based medical information retrieval frameworks.

show abstract

“…Considering that experts use generally a more specific vocabulary [21,23], we analysed the expertise level Expertise(U j , Q) of each user U j with respect to query Q using relevance feedback expressed through their respective TREC runs. For this purpose, we estimate the average specificity of the selected document set D S Q (Uj ) using the specificity indicator P spec [14] for search session S Q related to query Q:…”

Section: Methodsmentioning

confidence: 99%

A Collaborative Document Ranking Model for a Multi-faceted Search

Soulier

Tamine

Bahsoun

2013

Information Retrieval Technology

View full text Add to dashboard Cite

Abstract. This paper presents a novel collaborative document ranking model which aims at solving a complex information retrieval task involving a multi-faceted information need. For this purpose, we consider a group of users, viewed as experts, who collaborate by addressing the different query facets. We propose a two-step algorithm based on a relevance feedback process which first performs a document scoring towards each expert and then allocates documents to the most suitable experts using the Expectation-Maximisation learning-method. The performance improvement is demonstrated through experiments using TREC interactive benchmark.

show abstract

Relationship between index term specificity and relevance judgment

Cited by 14 publications

References 6 publications

Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance

Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance

Analysis of biomedical and health queries: Lessons learned fromTRECandCLEFevaluation benchmarks

A Collaborative Document Ranking Model for a Multi-faceted Search

Contact Info

Product

Resources

About