Evaluating retrieval performance given database and query characteristics: Analytic determination of performance surfaces

Abstract

An analytic method of information retrieval and filtering evaluation can quantitatively predict the expected number of documents examined in retrieving a relevant document. It also allows researchers and practitioners to qualitatively understand how varying different estimates of query parameter values affects retrieval performance. The incorporation of relevance feedback to increase our knowledge about the parameters of relevant documents and the robustness of parameter estimates is modeled. Single term and two term independence models, as well as a complete term dependence model, are developed. An economic model of retrieval performance may be used to study the effects of database size and to provide analytic answers to questions comparing retrieval from small and large databases, as well as questions about the number of terms in a query. Results are presented as a performance surface, a three dimensional graph showing the effects of two independent variables on performance.

Human–Computer Interaction Research in Information Retrieval

“…Based on the binary approach to relevant ranked information objects, Losee [101,102] developed average search length (ASL), to indicate the expected position of a relevant information object in the ranked list of information objects. Based on the binary approach to relevant ranked information objects, Losee [101,102] developed average search length (ASL), to indicate the expected position of a relevant information object in the ranked list of information objects.…”

Text windows and phrases differing by discipline, location in document, and syntactic structure

“…The experimental performance of retrieval systems that use terms clustered into windows has been studied (Akers, 1995;Haas & He, 1993;Haas & Losee, 1994;Losee, 1994). Performance of retrieval systems using groups of terms that are statistically related may be computed analytically rather than experimentally (Losee, 1995a(Losee, , 1996a.…”

“…We hope to continue the study of syntactic groupings for the automatic identi cation of key terms and phrases in documents, as well as to isolate those subject bearing parts of documents for use in systems that summarize documents. Additionally, allowing us to describe more important parts of documents will allow us to focus the analytic study of retrieval and ltering system performance (Losee, 1988(Losee, , 1995a(Losee, , 1996a on those parameters likely to have the greatest impact on performance.…”