A Hybrid Ontology and Visual-based Retrieval Model for Cultural Heritage Multimedia Collections

Abstract

Abstract. Nowadays, an increasingly growing demand for advanced multimedia search engines is arising, as huge amounts of digital visual content are becoming available. The contribution of this paper is the introduction of a hybrid multimedia retrieval model accompanied by the presentation of a search engine that is capable of retrieving visual content from cultural heritage multimedia libraries as in three modes: (i) based on their semantic annotation with the help of an ontology; (ii) based on the visual features with a view to finding similar content; and (iii) based on the combination of these two strategies in order to produce recommendations. To achieve this, the retrieval model is composed of two different parts, a low-level visual feature analysis and retrieval and a high-level ontology infrastructure. The main novelty is the way in which these two co-operate transparently during the evaluation of a single query in a hybrid fashion, making recommendations to the user and retrieving content that is both visually and semantically similar. A search engine has been developed implementing this model which is capable of searching through digital libraries of cultural heritage collections, and indicative examples are discussed, along with insights into its performance.

Ontologies on trial: the lesson ofMauricev.Judd(New York, 1818)

“…For sure, applications need not necessarily be what law enforcement agencies need. For example, there are applications to the cultural heritage: Vrochidis et al (2008) reported about a hybrid ontology and visual-based retrieval model for cultural heritage multimedia collections. Image processing is one of the areas of application of ontologies: Anouncia and Saravanan (2007) discussed ontology-based process plan generation for image processing.…”

Intangible Cultural Heritage and New Technologies: Challenges and Opportunities for Cultural Preservation and Development

“…In [53] and [54] computer vision techniques are employed in order to automatically classify archaeological pottery sherds. Lastly, a Computer Vision technique is also used in [55] where the authors present a search engine for retrieving cultural heritage multimedia content.…”

Utilizing Implicit User Feedback to Improve Interactive Video Retrieval

“…The results (i.e., the average weights for each action) of this survey are presented in Table 1. Using (4) and (5) and the defined values for g, we are able to construct the weighted graph. Figure 5 illustrates the weighted graph that is produced after processing the action graph of Figure 4 according to the aforementioned methodology.…”

“…Based on the audiovisual information, it is possible to extract low level features [2] for each shot; however, due to the well known problem of the semantic gap, it is difficult to associate them with human understandable concepts. Combination and fusion of heterogeneous information (e.g., visual, textual, and motion) have been considered as the first step towards the solution of this problem and promising results have been presented both in image and video retrieval domains [3][4][5][6]. However, the progress in the representation 2 Advances in Multimedia and indexing of the multimedia content has not yet managed to overcome the semantic gap.…”