A Weighted Topic Model Learned From Local Semantic Space for Automatic Image Annotation

Song, Haiyu; Wang, Pengjie; Jian, Yu; Wei, Li; Xue, Bo; Wu, Gang

doi:10.1109/access.2020.2989200

Cited by 18 publications

(8 citation statements)

References 28 publications

(54 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The JEC model utilizes various low-level image features and a simple combination of basic distance measures to find the nearest neighbors of a given image. It creates a family of very simple and intuitive baseline methods for image annotation [20]. Guillaumin et al presented the TagProp [12] method, which learns the weight of each feature group and uses the label relevance prediction to annotate images [2].…”

Section: Nearest Neighbor Based Modelsmentioning

confidence: 99%

“…Besides per-label metrics, more and more researchers adopt per-image metrics (also including precision, recall, and F1-measure) to evaluate annotation performance [9]- [11], [20], [30], [37], the per-label metrics are biased toward infrequent labels because making them correct could have a very significant impact on final accuracy [10]. These values are averaged over all the images in the test dataset to get This work is licensed under a Creative Commons Attribution 4.0 License.…”

Section: B Evaluation Metricsmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Image Annotation by Sequentially Learning From Multi-Level Semantic Neighborhoods

Zhang

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

Automatic image annotation is a key technology in image understanding and pattern recognition, and is becoming increasingly important in order to annotate large-scale images. In the past decade, the nearest neighbor model-based AIA methods have been proved to be the most successful in all classical models. This model has four major challenges including semantic gap, label-imbalance, wider range labels, and weak-labeling. In this paper, we propose a novel annotation model based on three-pass KNN (k-Nearest Neighbor) to address the aforementioned challenges. The key idea is to identify appropriate neighbors at each pass KNN. In the first pass KNN, we identify the several most relevant categories based on label feature rather than visual feature as traditional models. In the second pass KNN, we determine the relevant images based on multi-modal (visual and textual label) embedding features. As the test image has not been annotated with any label, we propose a pre-annotation strategory before image annotation to improve the semantic level. In the third pass KNN, we capture relevant labels from semantically and visually similar images and propagate them to the given unlabeled image. In contrast with traditional nearest neighbor based methods, our method can inherently alleviate the problems of semantic gap, label-imbalance, and wider range labels. In addition, to alleviate the issue of weak-labeling, we propose label refinement for training images. Extensive experiments on three classical benchmark datasets demonstrate that the proposed method significantly outperforms the state-of-the-art in terms of per-label and per-image metrics.INDEX TERMS Automatic image annotation, semantic gap, nearest neighbor, weak-labeling.

show abstract

Section: Nearest Neighbor Based Modelsmentioning

confidence: 99%

Section: B Evaluation Metricsmentioning

confidence: 99%

Automatic Image Annotation by Sequentially Learning From Multi-Level Semantic Neighborhoods

Zhang

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…The work of Song et al [17] also describes automated image annotation. Their approach includes the use of an intermediate layer in a neural network to extract data from images more accurately.…”

Section: Related Workmentioning

confidence: 99%

DeepScanner: a Robotic System for Automated 2D Object Dataset Collection with Annotations

Ilin¹,

Kalinov²,

Karpyshev³

et al. 2021

Preprint

View full text Add to dashboard Cite

In the proposed study, we describe the possibility of automated dataset collection using an articulated robot. The proposed technology reduces the number of pixel errors on a polygonal dataset and the time spent on manual labeling of 2D objects. The paper describes a novel automatic dataset collection and annotation system, and compares the results of automated and manual dataset labeling. Our approach increases the speed of data labeling 240-fold, and improves the accuracy compared to manual labeling 13-fold. We also present a comparison of metrics for training a neural network on a manually annotated and an automatically collected dataset.Index terms -Dataset collection, automated data annotation, image segmentation, industrial robot.

show abstract

“…These topic models are trained in the entire image database; therefore, they could not be applied to a larger image database because of expensive spacetime overheads. The local learning-based PLSA (LL-PLSA) approach, which models from a fixed number of neighbor images rather than the entire image database, significantly improves performance and can scale to large-scale image databases [33].…”

Section: E Topic Model Approachesmentioning

confidence: 99%

“…Recently, some researchers have pointed out that the perlabel metrics are biased toward infrequent labels because making them correct could have a very significant impact on final accuracy [25]. Therefore, they propose per-image metrics (sometimes called as overall metrics) to evaluate annotation performance [1], [5], [14], [25], [29], [32], [33]. These values are averaged over all the images in the test dataset to get average (percentage) per-image precision (P rec I ) and average per-label recall (Rec I ), respectively.…”

Section: B Evaluation Metricsmentioning

confidence: 99%

An Efficient and Effective Model Based on Mean Positive Examples for Social Image Annotation

Song

Jian

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

Nowadays, with the rapid growth of imaging and social network, huge volumes of image data are produced and shared on social media. Social image annotation has been an important and challenging task in the fields of computer vision and machine learning, which can facilitate large-scale image retrieval, indexing, and management. The four most challenges of social image annotation are semantic gap, tag refinement, label-imbalance, and annotation efficiency. To address these issues, we propose an efficient and effective annotation method based on the Mean of Positive Examples (MPE) corresponding to each label. First, we refine user-provided noisy tags by our proposed local smoothing process, and consider the refined tags as key features in contrast to the previous methods that consider them as side information, which significantly improves annotation performance. Second, we propose a weighted trans-media similarity measure method that fuses all modality information in identifying proper neighbors, which promotes the semantic level and eases image annotation. Third, our MPE model gives equal importance to all labels, thus, improving the annotation performance of infrequent labels without sacrificing that of frequent labels. Fourth, our MPE model can dramatically decrease space-time overheads, since the time cost of annotating an image is unaffected by the size of the training image dataset, but relying on the size of label vocabulary. Therefore, our proposed method can be applied to real-world large-scale online social image repositories. Extensive experiments on both benchmark datasets demonstrate the effectiveness and efficiency of our MPE model.

show abstract

A Weighted Topic Model Learned From Local Semantic Space for Automatic Image Annotation

Cited by 18 publications

References 28 publications

Automatic Image Annotation by Sequentially Learning From Multi-Level Semantic Neighborhoods

Automatic Image Annotation by Sequentially Learning From Multi-Level Semantic Neighborhoods

DeepScanner: a Robotic System for Automated 2D Object Dataset Collection with Annotations

An Efficient and Effective Model Based on Mean Positive Examples for Social Image Annotation

Contact Info

Product

Resources

About