Knowledge representation and semantic annotation of multimedia content

Abstract

Knowledge representation and annotation of multimedia documents typically have been pursued in two different directions. Previous approaches have focused either on low level descriptors, such as dominant color, or on the semantic content dimension and corresponding manual annotations, such as person or vehicle. In this paper, we present a knowledge infrastructure and a experimentation platform for semantic annotation to bridge the two directions. Ontologies are being extended and enriched to include low-level audiovisual features and descriptors. Additionally, we present a tool that allows for linking low-level MPEG-7 visual descriptions to ontologies and annotations. This way we construct ontologies that include prototypical instances of high-level domain concepts together with a formal specification of the corresponding visual descriptors. This infrastructure is exploited by a knowledge-assisted analysis framework that may handle problems like segmentation, tracking, feature extraction and matching in order to classify scenes, identify and label objects, thus automatically create the associated semantic metadata.

A learning approach to semantic image analysis

“…We compared the performance of our method with the method described in [3]. It must be noted that for the numerical evaluation, any object present in the examined image test set that was not included in the domain ontology concept definitions, e.g.…”

Using Fuzzy DLs to Enhance Semantic Image Analysis

“…Under these considerations, using formal languages to represent mappings between feature values and domain entities, such as in [4], where colour, texture and shape values are mapped to tumour types, or [10], where colour and shape values are mapped to natural objects, may be significant for purposes of sharing and reusing knowledge, but it does not leave much opportunities for utilising reasoning in terms of intelligence through computational means. It is not simply a matter of the limited datatype support provided by ontology languages such as RDFS 3 and OWL 4 , but because of the non logical nature of the problem at hand, i.e.…”

“…In the approaches addressing the transition from perceptual features to conceptual entities, thresholds and allowed ranges regarding the values of the considered features are used, i.e. ambiguity is treated as a separate aspect from the domain semantics [5,4,9,10]. On the other hand, in the approaches that focus more on the utilisation of semantics and inference for the purpose of acquiring descriptions of higher complexity, uncertainty is not taken into consideration at all [8,11].…”

Formal and graphical annotations for digital objects

“…for instance, [1], [17], [16], [14], [11], [20], [15]). The tools presented in this paper do not consider all the tasks an annotating tool should manage.…”