A New Baseline for Image Annotation

Makadia, Ameesh; Pavlović, Vladimir; Kumar, Sanjiv

doi:10.1007/978-3-540-88690-7_24

Cited by 346 publications

(391 citation statements)

References 19 publications

(21 reference statements)

Supporting

Mentioning

382

Contrasting

Order By: Relevance

“…More recently Makadia et al [28] have proposed a new, simple set of baseline image features for image annotation problems. They also proposed a simple technique to combine distance computations to create a nearest neighbor classifier suitable for baseline experiments.…”

Section: Image Features For Annotationmentioning

confidence: 99%

“…In a comparison of four distance measures (KL-divergence, a χ 2 statistic, L 1 -distance, and L 2 -distance) on the Corel dataset, [28] found that L 1 performed the best for RGB and HSV while the KL-divergence was better suited for LAB.…”

Section: Image Features For Annotationmentioning

confidence: 99%

“…Rather than constructing classifiers for a fixed set of labels, our approach operates on a much larger pool of words that are drawn from existing annotations on a video corpus. We extend the features used in the JEC method of image annotation [28] for use with video shots, and use a distributed spill-tree to inform the model of neighboring shots within the corpus. Each step of our approach is implemented in MapReduce for scalability and efficiency.…”

Section: A Probabilistic Model For Label Transfermentioning

confidence: 99%

“…Spill trees operate on data points within relatively low dimensional vector space, and this motivates our choice of feature representation for shot matching. We opt to use modified versions of the JEC features [28] which were discussed in Section 2.1. These image features include LAB and HSV global color histograms, and the Haar and Gabor wavelets.…”

Section: Nearest Neighbor For Scalabilitymentioning

confidence: 99%

“…This 1-nn approach, while simplistic, has been shown to provide state of the art results for image annotation [28]. Taking the idea further, the single-best annotation could be selected for each query shot individually, rather than for a complete query video comprised of a collection of shots.…”

Section: Alternative Modelsmentioning

confidence: 99%

See 4 more Smart Citations

YouTube Scale, Large Vocabulary Video Annotation

Morsillo

Mann

Pal

2010

Video Search and Mining

View full text Add to dashboard Cite

As video content on the web continues to expand, it is increasingly important to properly annotate videos for effective search and mining. While the idea of annotating static imagery with keywords is relatively well known, the idea of annotating videos with natural language keywords to enhance search is an important emerging problem with great potential to improve the quality of video search. However, leveraging web-scale video datasets for automated annotation also presents new challenges and requires methods specialized for scalability and efficiency. In this chapter we review specific, state of the art techniques for video analysis, feature extraction and classification suitable for extremely large scale automated video annotation. We also review key algorithms and data structures that make truly large scale video search possible. Drawing from these observations and insights, we present a complete method for automatically augmenting keyword annotations to videos using previous annotations for a large collection of videos. Our approach is designed explicitly to scale to YouTube sized datasets and we present some experiments and analysis for keyword augmentation quality using a corpus of over 1.2 million YouTube videos. We demonstrate how the automated annotation of webscale video collections is indeed feasible, and that an approach combining visual features with existing textual annotations yields better results than unimodal models.

show abstract

Section: Image Features For Annotationmentioning

confidence: 99%

Section: Image Features For Annotationmentioning

confidence: 99%

Section: A Probabilistic Model For Label Transfermentioning

confidence: 99%

Section: Nearest Neighbor For Scalabilitymentioning

confidence: 99%

Section: Alternative Modelsmentioning

confidence: 99%

See 3 more Smart Citations

YouTube Scale, Large Vocabulary Video Annotation

Morsillo

Mann

Pal

2010

Video Search and Mining

View full text Add to dashboard Cite

show abstract

Tag-based social image retrieval: An empirical evaluation

Sun¹,

Bhowmick²,

Nguyen³

et al. 2011

J. Am. Soc. Inf. Sci.

View full text Add to dashboard Cite

Tags associated with social images are valuable information source for superior image search and retrieval experiences. Although various heuristics are valuable to boost tag-based search for images, there is a lack of general framework to study the impact of these heuristics. Specifically, the task of ranking images matching a given tag query based on their associated tags in descending order of relevance has not been well studied. In this article, we take the first step to propose a generic, flexible, and extensible framework for this task and exploit it for a systematic and comprehensive empirical evaluation of various methods for ranking images. To this end, we identified five orthogonal dimensions to quantify the matching score between a tagged image and a tag query. These five dimensions are: (i) tag relatedness to measure the degree of effectiveness of a tag describing the tagged image; (ii) tag discrimination to quantify the degree of discrimination of a tag with respect to the entire tagged image collection; (iii) tag length normalization analogous to document length normalization in web search; (iv) tag-query matching model for the matching score computation between an image tag and a query tag; and (v) query model for tag query rewriting. For each dimension, we identify a few implementations and evaluate their impact on NUS-WIDE dataset, the largest humanannotated dataset consisting of more than 269K tagged images from Flickr. We evaluated 81 single-tag queries and 443 multi-tag queries over 288 search methods and systematically compare their performances using standard metrics including Precision at top-K, Mean Average Precision (MAP), Recall, and Normalized Discounted Cumulative Gain (NDCG).

show abstract

Improving image annotation via ranking‐oriented neighbor search and learning‐based keyword propagation

Cui

Lian

et al. 2014

Asso for Info Science & Tech

View full text Add to dashboard Cite

Automatic image annotation plays a critical role in modern keyword-based image retrieval systems. For this task, the nearest-neighbor-based scheme works in two phases: first, it finds the most similar neighbors of a new image from the set of labeled images; then, it propagates the keywords associated with the neighbors to the new image. In this article, we propose a novel approach for image annotation, which simultaneously improves both phases of the nearest-neighbor-based scheme. In the phase of neighbor search, different from existing work discovering the nearest neighbors with the predicted distance, we introduce a ranking-oriented neighbor search mechanism (RNSM), where the ordering of labeled images is optimized directly without going through the intermediate step of distance prediction. In the phase of keyword propagation, different from existing work using simple heuristic rules to select the propagated keywords, we present a learning-based keyword propagation strategy (LKPS), where a scoring function is learned to evaluate the relevance of keywords based on their multiple relations with the nearest neighbors. Extensive experiments on the Corel 5K data set and the MIR Flickr data set demonstrate the effectiveness of our approach.

show abstract

A New Baseline for Image Annotation

Cited by 346 publications

References 19 publications

YouTube Scale, Large Vocabulary Video Annotation

YouTube Scale, Large Vocabulary Video Annotation

Tag-based social image retrieval: An empirical evaluation

Improving image annotation via ranking‐oriented neighbor search and learning‐based keyword propagation

Contact Info

Product

Resources

About