Abstract. Annotations of multimedia documents typically have been pursued in two different directions. Either previous approaches have focused on low level descriptors, such as dominant color, or they have focused on the content dimension and corresponding annotations, such as person or vehicle. In this paper, we present a software environment to bridge between the two directions. M-OntoMat-Annotizer allows for linking low level MPEG-7 visual descriptions to conventional Semantic Web ontologies and annotations. We use M-OntoMatAnnotizer in order to construct ontologies that include prototypical instances of high-level domain concepts together with a formal specification of corresponding visual descriptors. Thus, we formalize the interrelationship of high-and low-level multimedia concept descriptions allowing for new kinds of multimedia content analysis and reasoning.

T h e S e m a n t i c W e b imprecise knowledge. More precisely, some applications deal with random information and events, others deal with imprecise and fuzzy knowledge, and still others deal with missing or distorted information-resulting in uncertainty. For example, in applications involving sensor readings, such measurements usually come with degrees of evidence; in applications like multimedia processing, object recognition might come with degrees of truth.To deal with uncertainty in the Semantic Web and its applications, many researchers have proposed extending OWL and the Description Logic (DL) formalisms with special mathematical frameworks. Researchers have proposed probabilistic, 1 possibilistic, 2 and fuzzy extensions, 3-5 among others. Researchers have studied fuzzy extensions most extensively, providing impressive results on semantics, reasoning algorithms, and implementations. Building on these results, we've created a fuzzy extension to OWL called Fuzzy OWL. Fuzzy OWL can capture imprecise and vague knowledge-for example, we can say that Athens is hot to a degree 0.8 rather than saying that Athens is either hot or not. Moreover, our reasoning platform, Fuzzy Reasoning Engine (FiRE), lets Fuzzy OWL capture and reason about such knowledge (see www.image.ece.ntua.gr/~nsimou).

Knowledge representation and annotation of multimedia documents typically have been pursued in two different directions. Previous approaches have focused either on low level descriptors, such as dominant color, or on the semantic content dimension and corresponding manual annotations, such as person or vehicle. In this paper, we present a knowledge infrastructure and a experimentation platform for semantic annotation to bridge the two directions. Ontologies are being extended and enriched to include low-level audiovisual features and descriptors. Additionally, we present a tool that allows for linking low-level MPEG-7 visual descriptions to ontologies and annotations. This way we construct ontologies that include prototypical instances of high-level domain concepts together with a formal specification of the corresponding visual descriptors. This infrastructure is exploited by a knowledge-assisted analysis framework that may handle problems like segmentation, tracking, feature extraction and matching in order to classify scenes, identify and label objects, thus automatically create the associated semantic metadata.

The effective management and exploitation of multimedia documents requires the extraction of the underlying semantics. Multimedia analysis algorithms can produce fairly rich, though imprecise information about a multimedia document which most of the times remains unexploited. In this paper we propose a methodology for semantic indexing and retrieval of images, based on techniques of image segmentation and classification combined with fuzzy reasoning. In the proposed knowledge-assisted analysis architecture a segmentation algorithm firstly generates a set of over-segmented regions. After that, a region classification process is employed to assign semantic labels using a confidence degree and simultaneously merge regions based on their semantic similarity. This information comprises the assertional component of a fuzzy knowledge base which is used for the refinement of mistakenly classified regions and also for the extraction of rich implicit knowledge used for global image classification. This knowledge about images is stored in a semantic repository permitting image retrieval and ranking.

No abstract