A Hybrid Model for Automatic Image Annotation

Abstract

In this work, we present a hybrid model (SVM-DMBRM) combining a generative and a discriminative model for the image annotation task. A support vector machine (SVM) is used as the discriminative model and a Discrete Multiple Bernoulli Relevance Model (DMBRM) is used as the generative model. The idea of combining both the models is to take advantage of the distinct capabilities of each model. The SVM tries to address the problem of poor annotation (images are not annotated with all relevant keywords), while the DMBRM model tries to address the problem of data imbalance (large variations in number of positive samples). Since DMBRM does not work well with high-dimensional data, a Latent Dirichlet Allocation (LDA) model is used to reduce the dimensionality of vector quantized features before using it. The hybrid model's results are comparable to or better than the state-of-the-art results on three standard datasets: Corel-5k, ESP-Game and IAPRTC-12.

Automatic Image Annotation using Deep Learning Representations

“…Most papers use one type of metric, but there is a small number of papers which use other metrics (For instance, In [8] the precision and recall are computed per word but they are computed only for non-zero recall words and their average over all non-zero recall words are reported). We use the standard (most widely reported type of evaluation where the recall and precision are computed per word and their average over all the words are reported [13,1,12,15,10,2]. We strictly adhere to computing the recall and precision per word (for all the words) and reporting their means over all the words, thus making it a fair comparison to the majority of the works in this area.…”

“…Request permissions from permissions@acm.org. visual appearance [10,2,4,13]. In these papers the goal is to predict a fixed number of tags for a given test image that accurately describe the visual content.…”

“…One variant, CCA-KNN significantly outperforms all the previously published results. We are able to achieve this without requiring any computationally expensive metric learning approaches as used by almost all successful models [13,1,12,15]. Some papers using CCA for combing image and text have been previously proposed [3,1], but the key differences are, we use CNN features as opposed to multiple handcrafted features (representing images) and we use word embedding vectors instead of binary vectors (representing tags).…”

Content-Based Image Retrieval for Multiple Objects Search

“…MCS is able to solve different classification tasks, as medical data classification [25], medical image retrieval [21], image annotation [19], and image retrieval [33]. In Multiple Queries for Image Retrieval (MQIR) systems [2] the end user would like to retrieve an interesting image by multiple queries (e.g., by more query images), which allows for a more expressive formulation of the query object, including different viewpoints and/or viewing conditions.…”

Large Scale Image Annotation via Deep Representation Learning and Tag Embedding Learning

“…In this paper, we address the issue of large scale image annotation, namely a large number of images with tags engaged. Many existing methods [2,19,14,10,3,15,1] for image annotation are established on small datasets, such as Corel5K [6], IAPRTC-12 [9] and ESP-game [18]. These datasets have only around 5000 to 20000 images, and these methods are difficult to be applied to large datasets.…”