Automatic image annotation using group sparsity

Zhang, Shaoting; Huang, Junzhou; Huang, Yuchi; Yu, Yang; Li, Hongsheng; Metaxas, Dimitris N.

doi:10.1109/cvpr.2010.5540036

Cited by 153 publications

(96 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also label interdependencies might result into corrupted distribution models. Another nearest-neighbour based method [5] tries to benefit from feature sparsity and clustering properties using a regularization based algorithm for feature selection. JEC [2] treats image annotation as retrieval.…”

Section: Related Workmentioning

confidence: 99%

Image Annotation Using Metric Learning in Semantic Neighbourhoods

Verma

Jawahar

2012

Lecture Notes in Computer Science

135

146

View full text Add to dashboard Cite

Abstract. Automatic image annotation aims at predicting a set of textual labels for an image that describe its semantics. These are usually taken from an annotation vocabulary of few hundred labels. Because of the large vocabulary, there is a high variance in the number of images corresponding to different labels ("class-imbalance"). Additionally, due to the limitations of manual annotation, a significant number of available images are not annotated with all the relevant labels ("weaklabelling"). These two issues badly affect the performance of most of the existing image annotation models. In this work, we propose 2PKNN, a two-step variant of the classical K-nearest neighbour algorithm, that addresses these two issues in the image annotation task. The first step of 2PKNN uses "image-to-label" similarities, while the second step uses "image-to-image" similarities; thus combining the benefits of both. Since the performance of nearest-neighbour based methods greatly depends on how features are compared, we also propose a metric learning framework over 2PKNN that learns weights for multiple features as well as distances together. This is done in a large margin set-up by generalizing a well-known (single-label) classification metric learning algorithm for multi-label prediction. For scalability, we implement it by alternating between stochastic sub-gradient descent and projection steps. Extensive experiments demonstrate that, though conceptually simple, 2PKNN alone performs comparable to the current state-of-the-art on three challenging image annotation datasets, and shows significant improvements after metric learning.

show abstract

Section: Related Workmentioning

confidence: 99%

Image Annotation Using Metric Learning in Semantic Neighbourhoods

Verma

Jawahar

2012

Lecture Notes in Computer Science

135

146

View full text Add to dashboard Cite

show abstract

“…It can be seen that our method consistently improves performance over the conventional one-vs-rest SVM. Also, it performs comparable or better than even the recently proposed annotation methods such as [6,8,23] (except for IAPRTC-12 dataset where its performance is inferior only to the best results of [8]). We also compare with two SVM-based models [1,9].…”

Section: Discussionmentioning

confidence: 99%

“…We use the same evaluation criteria as being used by previous methods [6,8,11,19,23]. Given a new sample, first we compute the score for each label using the corresponding classifier, and then assign it the five top-scoring labels.…”

Section: Discussionmentioning

confidence: 99%

“…This has applications in several tasks such as image retrieval, object recognition, robot navigation, etc. Hence this has emerged as an important research area during the last decade [2,5,6,8,11,19,23]. In annotation datasets with large vocabularies of few hundred or more labels, there exist three practical issues: (a) Incomplete-labeling: The training samples are not exhaustively tagged with all relevant labels from vocabulary.…”

Section: Introductionmentioning

confidence: 99%

“…Among the image annotation models being proposed in the past, generative or nearestneighbour (NN)-based models [5,8,11,19,23] have particularly been shown to be successful for large vocabulary datasets such as Corel-5k [4], ESP Game [20] and IAPRTC-12 [7]. The reason behind this is that in NN-based models, given a sample, the labels that are not present in the ground-truth of its neighbouring samples are simply ignored, rather than being considered as negative.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Exploring SVM for Image Annotation in Presence of Confusing Labels

Verma¹,

Jawahar²

2013

Procedings of the British Machine Vision Conference 2013

View full text Add to dashboard Cite

We address the problem of automatic image annotation in large vocabulary datasets. In such datasets, for a given label, there could be several other labels that act as its confusing labels. Three possible factors for this are (i) incomplete-labeling ("cars" vs. "vehicle"), (ii) label-ambiguity ("flowers" vs. "blooms"), and (iii) structural-overlap ("lion" vs. "tiger"). While previous studies in this domain have mostly focused on nearest-neighbour based models, we show that even the conventional one-vs-rest SVM significantly outperforms several benchmark models. We also demonstrate that with a simple modification in the hinge-loss of SVM, it is possible to significantly improve its performance. In particular, we introduce a tolerance-parameter in the hinge-loss. This makes the new model more tolerant against the errors in the classification of samples tagged with confusing labels as compared to other samples. This tolerance parameter is automatically determined using visual similarity and dataset statistics. Experimental evaluations demonstrate that our method (referred to as SVM with Variable Tolerance or SVM-VT) shows promising results on the task of image annotation on three challenging datasets, and establishes a baseline for such models in this domain.

show abstract

A Modality Converting Approach for Image Annotation to Overcome the Inconsistent Labels in Training Data

Suzuki

Ikeda

2021

Pattern Recognition. ICPR International Workshops and Challenges

View full text Add to dashboard Cite

Automatic image annotation using group sparsity

Cited by 153 publications

References 26 publications

Image Annotation Using Metric Learning in Semantic Neighbourhoods

Image Annotation Using Metric Learning in Semantic Neighbourhoods

Exploring SVM for Image Annotation in Presence of Confusing Labels

A Modality Converting Approach for Image Annotation to Overcome the Inconsistent Labels in Training Data

Contact Info

Product

Resources

About