Query-Adaptive Fusion for Multimodal Search

Kennedy, Lyndon; Chang, Shih-Fu; Natsev, Apostol

doi:10.1109/jproc.2008.916345

Cited by 36 publications

(22 citation statements)

References 41 publications

(52 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, a query-adaptive multimodal search strategy can be employed to provide the most useful retrieval results [Kennedy et al 2008]. Some multimedia video search models have been proposed recently, differing in the fusion strategies of search modalities.…”

Section: Content-based Retrievalmentioning

confidence: 99%

Vlogging

et al. 2010

View full text Add to dashboard Cite

In recent years, blogging has become an exploding passion among Internet communities. By combining the grassroots blogging with the richness of expression available in video, videoblogs (vlogs for short) will be a powerful new media adjunct to our existing televised news sources. Vlogs have gained much attention worldwide, especially with Google's acquisition of YouTube. This article presents a comprehensive survey of videoblogging (vlogging for short) as a new technological trend. We first summarize the technological challenges for vlogging as four key issues that need to be answered. Along with their respective possibilities, we give a review of the currently available techniques and tools supporting vlogging, and envision emerging technological directions for future vlogging. Several multimedia technologies are introduced to empower vlogging technology with better scalability, interactivity, searchability, and accessability, and to potentially reduce the legal, economic, and moral risks of vlogging applications. We also make an in-depth investigation of various vlog mining topics from a research perspective and present several incentive applications such as user-targeted video advertising and collective intelligence gaming. We believe that vlogging and its applications will bring new opportunities and drives to the research in related fields.

show abstract

Section: Content-based Retrievalmentioning

confidence: 99%

Vlogging

et al. 2010

View full text Add to dashboard Cite

show abstract

“…As we did not observe any significant impact from the optimisation of these values, we decided to leave them at neutral values of N = 10,000 and λ = 0.5. Although it is not the focus of this work, we tested a number of basic multimodal fusion techniques (see [14] for an overview) to dynamically set the λ parameter, but we did not find any significant improvement. In future work, we shall explore the application of more elaborated techniques, which could help on the combination of the different external KSs.…”

Section: Improving Manual Visual Examples With External Knowledgementioning

confidence: 93%

Exploiting semantics on external resources to gather visual examples for video retrieval

Vallet

Cantador

Jose

2012

Int J Multimed Info Retr

View full text Add to dashboard Cite

With the huge and ever rising amount of video content available on the Web, there is a need to facilitate video retrieval functionalities on very large collections. Most of the current Web video retrieval systems rely on manual textual annotations to provide keyword-based search interfaces. These systems have to face the problems that users are often reticent to provide annotations, and that the quality of such annotations is questionable in many cases. An alternative commonly used approach is to ask the user for an image example, and exploit the low-level features of the image to find video content whose keyframes are similar to the image. In this case, the main limitation is the so-called semantic gap, which consists of the fact that low-level image features often do not match with the real semantics of the videos. Moreover, this approach may be a burden to the user, as it requires finding and providing the system with relevant visual examples. Aiming to address this limitation, in this paper, we present a hybrid video retrieval technique that automatically obtains visual examples by performing textual searches on external knowledge sources, such as DBpedia, Flickr and Google Images, which have different coverage and structure characteristics. Our approach exploits the semantics underlying the above knowledge sources to address the semantic gap problem. We have conducted evaluations to assess the quality of visual examples retrieved from the above external knowledge sources.

show abstract

“…Although it is not the focus of this work, we tried a number of basic multimodal fusion techniques (see [11] for an overview) to dynamically set the ߣ parameter, but we did not find any significant improvement. In future work we will explore the application of more elaborated techniques, which could help in the combination of the different external KSs.…”

Section: Improving Manual Visual Examples With External Knowledgementioning

confidence: 96%

“…DBPedia is an ontology which stores structured information obtained from Wikipedia, and, making use of Semantic Web technologies, links that information with other KSs, such as OpenCyc, WordNet or DBLP, among others 11 .…”

Section: Dbpediamentioning

confidence: 99%

Exploiting external knowledge to improve video retrieval

Vallet

Cantador

Jose

2010

Proceedings of the International Conference on Multimedia Information Retrieval

View full text Add to dashboard Cite

Most video retrieval systems are multimodal, commonly relying on textual information, low-and high-level semantic features extracted from query visual examples. In this work, we study the impact of exploiting different knowledge sources in order to automatically retrieve query visual examples relevant to a video retrieval task. Our hypothesis is that the exploitation of external knowledge sources can help on the identification of query semantics as well as on improving the understanding of video contents.We propose a set of techniques to automatically obtain additional query visual examples from different external knowledge sources, such as DBPedia, Flickr and Google Images, which have different coverage and structure characteristics. The proposed strategies attempt to exploit the semantics underlying the above knowledge sources to reduce the ambiguity of the query, and to focus the scope of the image searches in the repositories.We assess and compare the quality of the images obtained from the different external knowledge sources when used as input of a number of video retrieval tasks. We also study how much they complement manually provided sets of examples, such as those given by TRECVid tasks.Based on our experimental results, we report which external knowledge source is more likely to be suitable for the evaluated retrieval tasks. Results also demonstrate that the use of external knowledge can be a good complement to manually provided examples and, when lacking of visual examples provided by a user, our proposed approaches can retrieve visual examples to improve the user's query.

show abstract

Query-Adaptive Fusion for Multimodal Search

Cited by 36 publications

References 41 publications

Vlogging

Vlogging

Exploiting semantics on external resources to gather visual examples for video retrieval

Exploiting external knowledge to improve video retrieval

Contact Info

Product

Resources

About