Can High-Level Concepts Fill the Semantic Gap in Video Retrieval? A Case Study With Broadcast News

Hauptmann, Alexander G.; Yan, Rong; Lin, Wei‐Hao; Christel, Michael G.; Wactlar, Howard D.

doi:10.1109/tmm.2007.900150

Cited by 180 publications

(122 citation statements)

References 17 publications

Supporting

Mentioning

119

Contrasting

Unclassified

Order By: Relevance

“…It appears that, in comparison to the 5,000 or so concepts predicted to be needed for sufficient performance in event detection [46], this number of high-level features begins to span the space of concepts reasonably well. Therefore, analogous sets of motion and audio concepts should further improve the overall performance.…”

Section: Summary and Discussionmentioning

confidence: 99%

Evaluating multimedia features and fusion for example-based event detection

Myers

Nallapati

Hout

et al. 2013

Machine Vision and Applications

View full text Add to dashboard Cite

Multimedia event detection (MED) is a challenging problem because of the heterogeneous content and variable quality found in large collections of Internet videos. To study the value of multimedia features and fusion for representing and learning events from a set of example video clips, we created SESAME, a system for video SEarch with Speed and Accuracy for Multimedia Events. SESAME includes multiple bag-of-words event classifiers based on single data types: low-level visual, motion, and audio features; highlevel semantic visual concepts; and automatic speech recognition. Event detection performance was evaluated for each event classifier. The performance of low-level visual and motion features was improved by the use of difference coding. The accuracy of the visual concepts was nearly as strong as that of the low-level visual features. Experiments with a number of fusion methods for combining the event detection scores from these classifiers revealed that simple fusion methods, such as arithmetic mean, perform as well as or better than other, more complex fusion methods. SESAME's

show abstract

Section: Summary and Discussionmentioning

confidence: 99%

Evaluating multimedia features and fusion for example-based event detection

Myers

Nallapati

Hout

et al. 2013

Machine Vision and Applications

View full text Add to dashboard Cite

show abstract

“…It is our hope that by establishing the synergy between them substantial progress is possible. Current detection rates are still low for many concepts, but there is hope [41] that even this limited detection accuracy with large numbers of concepts will be sufficient for substantial help with concept-based video retrieval.…”

Section: Discussionmentioning

confidence: 99%

A Multi-Pronged Approach to Improving Semantic Extraction of News Video

Hauptmann

Chen

Christel

et al. 2009

J Sign Process Syst Sign Image Video Technol

Self Cite

View full text Add to dashboard Cite

In this paper we describe a multi-strategy approach to improving semantic extraction from news video. Experiments show the value of careful parameter tuning, exploiting multiple feature sets and multilingual linguistic resources, applying text retrieval approaches for image features, and establishing synergy between multiple concepts through undirected graphical models. We present a discriminative learning framework called Multi-concept Discriminative Random Field (MDRF) for building probabilistic models of video semantic concept detectors by incorporating related concepts as well as the low-level observations. The model exploits the power of discriminative graphical models to simultaneously capture the associations of concept with observed data and the interactions between related concepts. Compared with previous methods, this model not only captures the co-occurrence between concepts but also incorporates the raw data observations into a unified framework. We also describe an approximate parameter estimation algorithm and present results obtained from the TRECVID 2006 data. No single approach, however, provides a consistently better result for all concept detection tasks, which suggests that extracting video semantics should exploit multiple resources and techniques rather than naively relying on a single approach

show abstract

“…In contrast to our work, in [16] and [37] each semantic concept is trained by using entire images or frames of videos. There is no sense of localized representation of meaningful object concepts in scenes.…”

Section: Related Workmentioning

confidence: 98%

“…The idea of using many object detectors as the basic representation of images is analogous to work in the multi-media community on applying a large number of "semantic concepts" to video and image annotation [16] and semantic indexing [37]. In contrast to our work, in [16] and [37] each semantic concept is trained by using entire images or frames of videos.…”

Section: Related Workmentioning

confidence: 99%

Objects as Attributes for Scene Classification

Lim

et al. 2012

Trends and Topics in Computer Vision

View full text Add to dashboard Cite

Abstract.Robust low-level image features have proven to be effective representations for a variety of high-level visual recognition tasks, such as object recognition and scene classification. But as the visual recognition tasks become more challenging, the semantic gap between low-level feature representation and the meaning of the scenes increases. In this paper, we propose to use objects as attributes of scenes for scene classification. We represent images by collecting their responses to a large number of object detectors, or "object filters". Such representation carries high-level semantic information rather than low-level image feature information, making it more suitable for high-level visual recognition tasks. Using very simple, off-the-shelf classifiers such as SVM, we show that this object-level image representation can be used effectively for high-level visual tasks such as scene classification. Our results are superior to reported state-of-the-art performance on a number of standard datasets.

show abstract

Can High-Level Concepts Fill the Semantic Gap in Video Retrieval? A Case Study With Broadcast News

Cited by 180 publications

References 17 publications

Evaluating multimedia features and fusion for example-based event detection

Evaluating multimedia features and fusion for example-based event detection

A Multi-Pronged Approach to Improving Semantic Extraction of News Video

Objects as Attributes for Scene Classification

Contact Info

Product

Resources

About