Query difficulty estimation via pseudo relevance feedback for image search

Abstract

Query difficulty estimation (QDE) attempts to automatically predict the performance of the search results returned for a given query. QDE has been widely investigated in text document retrieval for many years. However, few research works have been explored in image retrieval. State-of-the-art QDE methods in image retrieval mainly investigate the statistical characteristics (coherence, robustness, etc.) of the returned images to derive a value for indicating the query difficulty degree. To the best of our knowledge, little research has been done to directly estimate the real retrieval performance of the search results, such as average precision, instead of only an indicator. In this paper, we propose a novel query difficulty estimation approach which automatically estimate the average precision of the image search results. Specifically, we first select a set of query relevant and query irrelevant images for each query via pseudo relevance feedback. Then an efficient and effective voting scheme is proposed to estimate the relevance label of each image in the search results. Based on the images' relevance labels, the average precision of the search results returned for the given query is derived. The experimental results on a benchmark image search dataset demonstrate the effectiveness of the proposed method.

iQPP: A Benchmark for Image Query Performance Prediction

“…The authors adapt the visual content representation to the class predicted in the first stage. Jia et al [33,34] introduced a post-retrieval predictor that divides the retrieved images into pseudo-positive and pseudo-negative via pseudo-relevance feedback. Next, a voting scheme is applied to label the images as relevant or not.…”

“…Most of the above studies, e.g. [33,34,45,70,71,74], predict the performance of text queries in CBIR. Our study is among the few works [42,46,68,72] studying the performance of image queries.…”

“…The importance of QPP has been widely recognized in text retrieval [3, 10-13, 18, 24-26, 35, 39, 40, 43, 44, 51, 54-56, 58, 63-65, 69, 76, 77], being an actively studied task nowadays [1, 2, 9, 14-16, 22, 32, 60]. However, in the context of content-based image retrieval (CBIR), the QPP task received comparably lower attention from the research community, with only a few studies addressing the topic [33,34,42,45,46,68,[70][71][72]74]. Furthermore, only a handful of papers [42,46,68,72] study query difficulty prediction in the queryby-example scenario, where the query is an image.…”