Evaluation of search results: A new approach

Abstract

In the present work, a new approach to construction of single‐valued measures for evaluation of search results is considered in detail. This approach is based first on the formation of certain vectors, and second on the use of functions allowing the evaluation of their closeness. Realization of this approach leads to construction of new, as well as widely‐known, single‐valued measures. A series of constructed measures has a property important in practice, leading to the creation of a mechanism which would allow the selection of the best search method (from a set of available methods) for every query. In this article, a general statement of this property, called the order preservation property, is given, and all considered single‐valued measures are analyzed for the existence of this property. Finally, problems of the legitimacy of the use of single‐valued measures in various specific situations are discussed in the article, and an approach to the solution of these problems is suggested. © 1997 John Wiley & Sons, Inc.

Search result list evaluation versus document evaluation: similarities and differences

“…the right pieces, the appropriateness of the articles, the directly relevant); and the completeness (all that is there or no missing information)" (p. 211). Voiskunskii (1997), on the other hand, found a "lack of a criterion for comparison of search results which could actually be used in practice" (p. 133). Rieh (2002) explored users' reliance on information quality and cognitive authority as judgment measures for selection and evaluation of information.…”

“…Search result list evaluation receives significantly less attention than document evaluation. Its research remains divided between coverage aspect of relevance (Park, 1994;Purgailis Parker and Johnson, 1990) and quality aspect of relevance (Rieh, 2002;Su, 1994;Voiskunskii, 1997). The majority of current research on evaluation remains focused on coverage aspect of relevance criteria within document evaluation (Bales and Wang, 2005;Barry, 1994Barry, , 1998Greisdorf, 2003;Schamber et al, 1990;Spink et al, 1998; although credibility (Fogg et al, 2003;Metzger et al, 2003;Xu and Chen, 2006) and other aspects of relevance criteria (Barry and Schamber, 1998;Soergel, 1998, 1999) occasionally appear.…”

A fuzzy-based method for improving recall values in recommender systems

“…This measure is also widely used in Information Retrieval and its calculation formula differs from author to author [5,10,13,15]. For the need of this work we have chosen the definition proposed in [15]:…”

Effectiveness of retrieval in similarity searches of chemical databases: a review of performance measures

“…Voiskunskii [19] has noted that similarity coefficients provide a simple and direct basis for the measurement of retrieval performance and demonstrates the use of the cosine coefficient to obtain the combined measure PR .…”