High-level semantic image annotation based on hot Internet topics

Abstract

Images are complex multimedia data that contain rich semantic information. Currently, most of image annotation algorithms are only annotating the object semantics of images. There are still many challenges on high-level semantic image annotation. The major issues are the lack of effective modeling method for the highlevel semantics of images and the lack of efficient dynamic update mechanism for the training set. To address these issues, we propose a high-level semantic annotation method based on hot Internet topics in this paper. There are two independent sub tasks in our method: dynamic update of the training set based on hot Internet topics and search-based image annotation. In the first sub task, we propose to model the abstract semantics of images based on three relationships: image-to-image similarity relationship, topic-to-topic co-occurrence relationship, and image-to-topic relevance relationship. Through the complex graph clustering, the hot Internet topics are extracted for images with consistent visual and semantic contents. Then the dynamic update mechanism will update the original training set with the new topics and images. It avoids the huge computing cost in traditional update methods and does not need to re-calculate the whole mapping relationship between the semantic concepts and visual features. In the second sub task, given a query image, it first searches for similar candidates in the annotated training set via visual features. Then the hypergraph modeling and spectral clustering are exploited to filter out the images with irrelevant semantics. The keywords will be extracted for annotation from the remaining images according to an annotation probability. Extensive experiments have been conducted and the results demonstrate that our algorithm could achieve better annotation performance than the state-of-the-art algorithms. And the update mechanism could extend the training set efficiently so that the coverage of the semantics in the training set wouldn't be obsolete.

Development and application of the semantic annotation framework for digital images

“…There are many existing studies on semantic annotation of images (Cheng and Wang, 2021; Latha et al , 2020; Wang and Zhu, 2021; Wang et al , 2015; Yang and Xu, 2021; Zhou and Jiang, 2021), among which the development of semantic models is the core. A semantic model for image annotation is a conceptual model obtained by conceptualizing and abstracting image content (Shi and Cai, 2004).…”

Considered Factors of Online News Based on Respondents’ Eye Activity Using Eye-Tracker Analysis

“…Media information is disseminated in various forms on the internet, and several examples include news text, images, and videos. The most substantial aspect is through pictures, because a picture says a thousand words, and millions of new photos are uploaded every day to the internet, which can be accessed by a lot of people [5]. Videos combined with text can attract people's attention and make them believe the information conveyed.…”