Towards a semantics representation framework for narrative images

Li, Xuhui; Wu, Yanqiu; Wang, Xiaoguang; Qian, Tieyun; Hong, Liang

doi:10.1108/el-09-2018-0187

Cited by 6 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An improved inpainting algorithm for repairing the Dunhuang murals was proposed, and it obtained good visual effects, as well as improved objective evaluation EL 40,3 values, such as the peak signal-to-noise ratio of the image (Chen et al, 2020a(Chen et al, , 2020b. As the Dunhuang murals are an asset of the world, many scholars have tried to research the flak prediction, semantic annotation, semantics representation, date prediction and inpainting of them (Li et al, 2018(Li et al, , 2019Sun et al, 2015;Wang et al, 2018Wang et al, , 2019. However, how to search for similar, relevant and diversified Dunhuang murals conveniently has been seldom researched.…”

Section: Literature Review 21 Dunhuang Cultural Heritagementioning

confidence: 99%

Constructing a mobile visual search framework for Dunhuang murals based on fine-tuned CNN and ontology semantic distance

Zeng

Sun

et al. 2022

View full text Add to dashboard Cite

Purpose Dunhuang murals are rich in cultural and artistic value. The purpose of this paper is to construct a novel mobile visual search (MVS) framework for Dunhuang murals, enabling users to efficiently search for similar, relevant and diversified images. Design/methodology/approach The convolutional neural network (CNN) model is fine-tuned in the data set of Dunhuang murals. Image features are extracted through the fine-tuned CNN model, and the similarities between different candidate images and the query image are calculated by the dot product. Then, the candidate images are sorted by similarity, and semantic labels are extracted from the most similar image. Ontology semantic distance (OSD) is proposed to match relevant images using semantic labels. Furthermore, the improved DivScore is introduced to diversify search results. Findings The results illustrate that the fine-tuned ResNet152 is the best choice to search for similar images at the visual feature level, and OSD is the effective method to search for the relevant images at the semantic level. After re-ranking based on DivScore, the diversification of search results is improved. Originality/value This study collects and builds the Dunhuang mural data set and proposes an effective MVS framework for Dunhuang murals to protect and inherit Dunhuang cultural heritage. Similar, relevant and diversified Dunhuang murals are searched to meet different demands.

show abstract

Section: Literature Review 21 Dunhuang Cultural Heritagementioning

confidence: 99%

Constructing a mobile visual search framework for Dunhuang murals based on fine-tuned CNN and ontology semantic distance

Zeng

Sun

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Different types of images require different information organization methods. Narrative images are common in the cultural heritage domain (Li et al, 2019): A story or event is represented through elements in an image rather than free text, and changes in settings and entities inside an image indicate the evolution of time and space. In addition to other explicit semantic units such as basic objects and actions, the "plot" becomes a structure linking an image and story texts and is a major semantic unit in DSA-CH.…”

Section: Narrative Image Annotation Model Of Cultural Heritage Imagesmentioning

confidence: 99%

Data modeling and evaluation of deep semantic annotation for cultural heritage images

Wang

Song

Liu

et al. 2021

Self Cite

View full text Add to dashboard Cite

PurposeTo meet the emerging demand for fine-grained annotation and semantic enrichment of cultural heritage images, this paper proposes a new approach that can transcend the boundary of information organization theory and Panofsky's iconography theory.Design/methodology/approachAfter a systematic review of semantic data models for organizing cultural heritage images and a comparative analysis of the concept and characteristics of deep semantic annotation (DSA) and indexing, an integrated DSA framework for cultural heritage images as well as its principles and process was designed. Two experiments were conducted on two mural images from the Mogao Caves to evaluate the DSA framework's validity based on four criteria: depth, breadth, granularity and relation.FindingsResults showed the proposed DSA framework included not only image metadata but also represented the storyline contained in the images by integrating domain terminology, ontology, thesaurus, taxonomy and natural language description into a multilevel structure.Originality/valueDSA can reveal the aboutness, ofness and isness information contained within images, which can thus meet the demand for semantic enrichment and retrieval of cultural heritage images at a fine-grained level. This method can also help contribute to building a novel infrastructure for the increasing scholarship of digital humanities.

show abstract

“…Evolution in a knowledge base usually focuses on three topics: the complementation of the attributes of a concept instance, the specialization of a coarsely defined concept and the generalization of similar concepts or instances (Li et al , 2019). Under the big data circumstance, the concepts should be combined with the data to be used in semantic retrieval and knowledge discovery.…”

Section: Scenario Of Knowledge Evolution On Video Semanticsmentioning

confidence: 99%

Towards evolutionary knowledge representation under the big data circumstance

Liu

Wang

et al. 2021

Self Cite

View full text Add to dashboard Cite

Purpose The purpose of this paper is to propose a graph-based representation approach for evolutionary knowledge under the big data circumstance, aiming to gradually build conceptual models from data. Design/methodology/approach A semantic data model named meaning graph (MGraph) is introduced to represent knowledge concepts to organize the knowledge instances in a graph-based knowledge base. MGraph uses directed acyclic graph–like types as concept schemas to specify the structural features of knowledge with intention variety. It also proposes several specialization mechanisms to enable knowledge evolution. Based on MGraph, a paradigm is introduced to model the evolutionary concept schemas, and a scenario on video semantics modeling is introduced in detail. Findings MGraph is fit for the evolution features of representing knowledge from big data and lays the foundation for building a knowledge base under the big data circumstance. Originality/value The representation approach based on MGraph can effectively and coherently address the major issues of evolutionary knowledge from big data. The new approach is promising in building a big knowledge base.

show abstract

Towards a semantics representation framework for narrative images

Cited by 6 publications

References 14 publications

Constructing a mobile visual search framework for Dunhuang murals based on fine-tuned CNN and ontology semantic distance

Constructing a mobile visual search framework for Dunhuang murals based on fine-tuned CNN and ontology semantic distance

Data modeling and evaluation of deep semantic annotation for cultural heritage images

Towards evolutionary knowledge representation under the big data circumstance

Contact Info

Product

Resources

About