Exploiting external knowledge to improve video retrieval

Abstract

Most video retrieval systems are multimodal, commonly relying on textual information, low-and high-level semantic features extracted from query visual examples. In this work, we study the impact of exploiting different knowledge sources in order to automatically retrieve query visual examples relevant to a video retrieval task. Our hypothesis is that the exploitation of external knowledge sources can help on the identification of query semantics as well as on improving the understanding of video contents.We propose a set of techniques to automatically obtain additional query visual examples from different external knowledge sources, such as DBPedia, Flickr and Google Images, which have different coverage and structure characteristics. The proposed strategies attempt to exploit the semantics underlying the above knowledge sources to reduce the ambiguity of the query, and to focus the scope of the image searches in the repositories.We assess and compare the quality of the images obtained from the different external knowledge sources when used as input of a number of video retrieval tasks. We also study how much they complement manually provided sets of examples, such as those given by TRECVid tasks.Based on our experimental results, we report which external knowledge source is more likely to be suitable for the evaluated retrieval tasks. Results also demonstrate that the use of external knowledge can be a good complement to manually provided examples and, when lacking of visual examples provided by a user, our proposed approaches can retrieve visual examples to improve the user's query.

Description and retrieval of the visual content of multimedia information based on intelligent techniques

“…Vallet et al [235] proposed a set of techniques to automatically obtain additional query visual examples from dierent external knowledge sources, such as DBPedia, Flickr and Google Images, which have dierent coverage and structure characteristics. The proposed strategies attempt to exploit the semantics underlying the above knowledge sources to reduce the ambiguity of the query, and to focus the scope of the image searches in the repositories.…”