Image annotation based on feature fusion and semantic similarity

Zhang, Xiaochun; Liu, Chuancai

doi:10.1016/j.neucom.2014.08.027

Cited by 21 publications

(13 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, we display all of the annotation outcomes based on the five categories. We make comparisons between our learning transfer model based on the label localization strategy and the traditional image annotation method based on multifeature fusion and semantic similarity in [ 27 ] and the Gaussian mixture model that considers cross-modal correlations in [ 28 ] (GMM-MB). We also conducted experimental comparisons in terms of precision between the feature fusion model proposed in [ 29 ] and the semantic extension model (SEM) proposed in [ 30 ], which use a CNN to extract features.…”

Section: Resultsmentioning

confidence: 99%

A localization strategy combined with transfer learning for image annotation

et al. 2021

View full text Add to dashboard Cite

This study aims to solve the overfitting problem caused by insufficient labeled images in the automatic image annotation field. We propose a transfer learning model called CNN-2L that incorporates the label localization strategy described in this study. The model consists of an InceptionV3 network pretrained on the ImageNet dataset and a label localization algorithm. First, the pretrained InceptionV3 network extracts features from the target dataset that are used to train a specific classifier and fine-tune the entire network to obtain an optimal model. Then, the obtained model is used to derive the probabilities of the predicted labels. For this purpose, we introduce a squeeze and excitation (SE) module into the network architecture that augments the useful feature information, inhibits useless feature information, and conducts feature reweighting. Next, we perform label localization to obtain the label probabilities and determine the final label set for each image. During this process, the number of labels must be determined. The optimal K value is obtained experimentally and used to determine the number of predicted labels, thereby solving the empty label set problem that occurs when the predicted label values of images are below a fixed threshold. Experiments on the Corel5k multilabel image dataset verify that CNN-2L improves the labeling precision by 18% and 15% compared with the traditional multiple-Bernoulli relevance model (MBRM) and joint equal contribution (JEC) algorithms, respectively, and it improves the recall by 6% compared with JEC. Additionally, it improves the precision by 20% and 11% compared with the deep learning methods Weight-KNN and adaptive hypergraph learning (AHL), respectively. Although CNN-2L fails to improve the recall compared with the semantic extension model (SEM), it improves the comprehensive index of the F1 value by 1%. The experimental results reveal that the proposed transfer learning model based on a label localization strategy is effective for automatic image annotation and substantially boosts the multilabel image annotation performance.

show abstract

Section: Resultsmentioning

confidence: 99%

A localization strategy combined with transfer learning for image annotation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Table 1, shows mean precision, recall and F 1 for the proposed methodology and the appropriate annotation recently presented three methods (IAGA -2014 [20], Feature fusion and semantic similarity-2014 [5], MLRank -2013 [21]). More areas have higher density than other areas in each image by weighting the edges of the graph.…”

Section: Experimental Results and Evaluationmentioning

confidence: 99%

“…[1] Great deal of research has been done in the field of image annotation that can be grouped into three models: probabilistic models, model-based on categories and models based on the nearest neighborhood. [5] Most probabilistic models [6,7,8,9] joint probability estimate on the image content and keywords. Model-based categories [10,11], the image annotation to be an issue with the supervisor behave category.…”

Section: Introductionmentioning

confidence: 99%

Automatic Image Annotation based on Dense Weighted Regional Graph

Boorjandi¹,

Ghobadi²,

Rashidi³

2017

ijacsa

View full text Add to dashboard Cite

Abstract-Automatic image annotation refers to create text labels in accordance with images' context automatically. Although, numerous studies have been conducted in this area for the past decade, existence of multiple labels and semantic gap between these labels and visual low-level features reduced its performance accuracy. In this paper, we suggested an annotation method, based on dense weighted regional graph. In this method, clustering areas was done by forming a dense regional graph of area classification based on strong fuzzy feature vector in images with great precision, as by weighting edges in the graph, less important areas are removed over time and thus semantic gap between low-level features of image and human interpretation of high-level concepts reduces much more. To evaluate the proposed method, COREL database, with 5,000 samples have been used. The results of the images in this database, show acceptable performance of the proposed method in comparison to other methods.

show abstract

“…Saliency can than be determined by ranking these nodes based on their similarities to background and foreground queries. In [21], a multi-feature fusion method was developed based on semantic similarity for image annotation.…”

Section: Introductionmentioning

confidence: 99%