Image annotation using clickthrough data

Tsikrika, Theodora; Diou, Christos; Vries, Arjen P. de; Delopoulos, Anastasios

doi:10.1145/1646396.1646415

Cited by 31 publications

(20 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…There are also works [23], [24], [25] that rely on the same principle assumption with our work, stating that users tend to contribute similar tags when faced with similar type of visual content. In [23] the authors are based on social data to introduce the concept of flickr distance.…”

Section: Related Workmentioning

confidence: 77%

“…They are not concerned with object detection but rather with concept detection modeled as a mixture/constellation of different object detectors. In the same lines, the work presented in [25] investigates inexpensive ways to generate annotated training samples for building concept classifiers. The authors utilize clickthrough data logged by retrieval systems that consist of the queries submitted by the users, together with the images from the retrieved results, that these users selected to click on in response to their queries.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Leveraging social media for scalable object detection

Chatzilari

Nikolopoulos

Patras

et al. 2012

Pattern Recognition

View full text Add to dashboard Cite

In this manuscript we present a method that leverages social media for the effortless learning of object detectors. We are motivated by the fact that the increased training cost of methods demanding manual annotation, limits their ability to easily scale in different types of objects and domains. At the same time, the rapidly growing social media applications have made available a tremendous volume of tagged images, which could serve as a solution for this problem. However, the nature of annotations (i.e. global level) and the noise existing in the associated information (due to lack of structure, ambiguity, redundancy, emotional tagging), prevents them from being readily compatible (i.e. accurate region level annotations) with the existing methods for training object detectors. We present a novel approach to overcome this deficiency by using the collective knowledge aggregated in social sites to automatically determine a set of image regions that can be associated with a certain object. We study theoretically and experimentally when the prevailing trends (in terms of appearance frequency) in visual and tag information space converge into the same object, and how this convergence is influenced by the number of utilized images and the accuracy of the visual analysis algorithms. Evaluation results show that although the models trained using leveraged social media are inferior to the ones trained manually, there are cases where the user contributed content can be successfully used to facilitate scalable and effortless learning of object detectors.

show abstract

Section: Related Workmentioning

confidence: 77%

Section: Related Workmentioning

confidence: 99%

Leveraging social media for scalable object detection

Chatzilari

Nikolopoulos

Patras

et al. 2012

Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…Smith & Ashman [21] study click-through data for image search and, consistently with Craswell et al, showed that it was 'considerably more accurate in general than document based search click-through data', although the reliability of the clicks was shown to be dependent on factors such as query type and the quality (in terms of precision) of the results shown. Tsikrika et al [24] have used image clicks to automatically create ground truth labels for training visual classifiers, while other work has combined click data and visual features for image ranking [30]. Other researchers have studied web image search interaction logs to understand how interaction with image search engines differs from general search, without exploring the utility of these interactions as implicit relevance information [10,2,17,19].…”

Section: Related Workmentioning

confidence: 99%

Leveraging User Interaction Signals for Web Image Search

O’Hare

Juan

Schifanella

et al. 2016

Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

User interfaces for web image search engine results differ significantly from interfaces for traditional (text) web search results, supporting a richer interaction. In particular, users can see an enlarged image preview by hovering over a result image, and an 'image preview' page allows users to browse further enlarged versions of the results, and to click-through to the referral page where the image is embedded. No existing work investigates the utility of these interactions as implicit relevance feedback for improving search ranking, beyond using clicks on images displayed in the search results page. In this paper we propose a number of implicit relevance feedback features based on these additional interactions: hover-through rate, 'converted-hover' rate, referral page click through, and a number of dwell time features. Also, since images are never self-contained, but always embedded in a referral page, we posit that clicks on other images that are embedded on the same referral webpage as a given image can carry useful relevance information about that image. We also posit that query-independent versions of implicit feedback features, while not expected to capture topical relevance, will carry feedback about the quality or attractiveness of images, an important dimension of relevance for web image search. In an extensive set of ranking experiments in a learning to rank framework, using a large annotated corpus, the proposed features give statistically significant gains of over 2% compared to a state of the art baseline that uses standard click features.

show abstract

“…This paper builds on our previous work [30] that investigated the effectiveness of the proposed approach. This extension presents further analysis and insights on the results of our experimental evaluation, with particular focus on the reliability of the search log based training samples and the effect of the levels of noise in these samples on the precision of the concept classifiers.…”

Section: Introductionmentioning

confidence: 99%

Reliability and effectiveness of clickthrough data for automatic image annotation

Tsikrika

Diou

Vries

et al. 2010

Multimed Tools Appl

Self Cite

View full text Add to dashboard Cite

Automatic image annotation using supervised learning is performed by concept classifiers trained on labelled example images. This work proposes the use of clickthrough data collected from search logs as a source for the automatic generation of concept training data, thus avoiding the expensive manual annotation effort. We investigate and evaluate this approach using a collection of 97,628 photographic images. The results indicate that the contribution of search log based training data is positive despite their inherent noise; in particular, the combination of manual and automatically generated training data outperforms the use of manual data alone. It is therefore possible to use clickthrough data to perform large-scale image annotation with little manual annotation effort or, depending on performance, using only the automatically generated training data. An extensive T. Tsikrika (B) · A. P.

show abstract

Image annotation using clickthrough data

Cited by 31 publications

References 19 publications

Leveraging social media for scalable object detection

Leveraging social media for scalable object detection

Leveraging User Interaction Signals for Web Image Search

Reliability and effectiveness of clickthrough data for automatic image annotation

Contact Info

Product

Resources

About