Search for Multi-modality Data in Digital Libraries

Abstract

Developing effective and efficient retrieval techniques for multimedia data is a challenging issue in building a digital library. Unlike most previously proposed retrieval approaches that focus on a specific media type, this paper presents 2M2Net as a seamless integration framework for retrieval of multi-modality data in digital libraries. As its specific approaches, a learning-fromelements strategy is devised for propagation of semantic descriptions, and a cross media search mechanism with relevance feedback is proposed for evaluation and refinement of user queries. Experiments conducted on a digital encyclopedia manifest the effectiveness and flexibility of our approaches.

Study of Cross-media Retrieval Technique Based on Ontology

“…(2) Second methods, they construct knowledge base for multimedia data. Literature [5] analyzed the multimedia data of digital library, indexed different type media separately, and to realize cross-media retrieval. But this method is mainly based on keywords, it has big workload and can not describe the information that multimedia include properly.…”

“…But this method is mainly based on keywords, it has big workload and can not describe the information that multimedia include properly. Therefore, this paper uses the knowledge base manner of literature [5], based on the content-based multimedia retrieval, organizing multimedia information by using ontology, analyzing the content and semanteme of multimedia, and achieving media span at semantic level.…”

Octopus

“…In the past few years, some works have investigated the integration of multi-modality data, usually between text and image, for better retrieval performance. For example, the iFind [17] system proposes a unified framework under which the semantic feature (text) and low-level features are combined for image retrieval, and the 2M2Net [23] system extends this framework to the retrieval of video and audio. WebSEEK system [21] extracts keywords from the surrounding text of image and videos, which is used as their indexes in the retrieval process.…”

Web-based multimedia retrieval: balancing out between common knowledge and personalized views

“…In this section, we discuss some implementation issues that are critical to the practical use of this approach. More details about 2M2Net can be found in [16].…”