Evaluation of retrieval performance is a crucial problem in content-based image retrieval CBIR. Many di erent methods for measuring the performance of a system have been created and used by researchers. This article discusses the advantages and shortcomings of the performance measures currently used. Problems such a s a commonimage database for performance comparisons and a means of getting relevance judgments or ground truth for queries are explained. The relationship between CBIR and information retrieval IR is made clear, since IR researchers have decades of experience with the evaluation problem. Many of their solutions can be used for CBIR, despite the di erences between the elds. Several methods used in text retrieval are explained. Proposals for performance measures and means of developing a standard test suite for CBIR, similar to that used in IR at the annual Text REtrieval Conference TREC, are presented.
Relevance feedback has been shown to be a very effective tool for enhancing retrieval results in text retrieval. In content-based image retrieval it is more and more frequently used and verv good results have been obtained. However, too much negative feedback may destroy a query as good features get negative weightings.This paper compares a variety of strategies for positive and negative feedback. The performance evaluation of feedback algorithms is a hard problem. To solve this, we obtain judgmentsfiom several users and employ an automated feedback scheme. We can then evaluate different techniques using the same judgments. Using automated feedback, the ability of a systeni to adapt to the user's needs can be measured very effectively. Our study highlights the utility of negative feedback, especially over severa1,feedback steps.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.