Complementary video and audio analysis for broadcast news archives

Wactlar, Howard D.; Hauptmann, Alexander G.; Christel, Michael G.; Houghton, R.; Olligschlaeger, Andreas

doi:10.1145/328236.328144

Cited by 26 publications

(14 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the Informedia system offers functionality, described in [278], that allows a user to initiate a query by specifying a location on a map. The system returns news stories containing references to places in this region.…”

Section: Query Entrymentioning

confidence: 99%

Automatic Summarization

Larson

2012

FNT in Information Retrieval

View full text Add to dashboard Cite

Section: Query Entrymentioning

confidence: 99%

Automatic Summarization

Larson

2012

FNT in Information Retrieval

View full text Add to dashboard Cite

“…The target of the Informedia project was first the cross-modal analysis of speech, language and images for digital video libraries (1994)(1995)(1996)(1997)(1998)(1999), and then the automatic summarization of information across multimedia documents (1999)(2000)(2001)(2002)(2003) [Wactlar et al, 1996[Wactlar et al, , 2000. In parallel, CMU's Interactive Systems Laboratory initiated a project on meeting record creation and access [Waibel et al, 2001].…”

Section: Research On Multimodal Human Interaction Analysismentioning

confidence: 99%

Multimodal signal processing for meetings: an introduction

Popescu-Belis

Carletta

2012

Multimodal Signal Processing

View full text Add to dashboard Cite

This book is an introduction to multimodal signal processing. In it, we use the goal of building applications that can understand meetings as a way to focus and motivate the processing we describe. Multimodal signal processing takes the outputs of capture devices running at the same time -primarily cameras and microphones, but also electronic whiteboards and pens -and automatically analyses them to make sense of what is happening in the space being recorded. For instance, these analyses might indicate who spoke, what was said, whether there was an active discussion, and who was dominant in it. These analyses require the capture of multimodal data using a range of signals, followed by a low-level automatic annotation of them, gradually layering up annotation until information that relates to user requirements is extracted.Multimodal signal processing can be done in real time, that is, fast enough to build applications that influence the group while they are together, or offline -not always but often at higher quality -for later review of what went on. It can also be done for groups that are all together in one space, typically an instrumented meeting room, or for groups that are in different spaces but use technology such as video-conferencing to communicate. The book thus introduces automatic approaches to capturing, processing and ultimately understanding human interaction in meetings, and describes the state-of-the-art for all technologies involved.Multimodal signal processing raises the possibility of a wide range of applications that help groups improve their interactions and hence their effectiveness between or during meetings. However, developing applications has required improvements in the technological state-of-theart in many arenas.The first comprises core technologies like audio and visual processing and recognition that tell us basic facts such as who was present and what words were said. On top of this information comes processing that begins to make sense of a meeting in human terms. Part of this is simply combining different sources of information into a record of who said what, when, and to whom, but it is often also useful, for instance, to apply models of group dynamics from the behavioral and social sciences in order to reveal how a group interacts, or to abstract and summarize the meeting content overall. Finding ways to integrate the varying analyses required for a particular meeting support application has been a major new challenge.Finally, moving from components that model and analyze multimodal human-to-human communication scenes to real-world applications has required careful user requirements capture,

show abstract

“…Similar to our approach, they also utilized news transcripts and external news articles. In particular, they used the extracted name lists harvested from news transcripts and external news articles to improve the accuracy of video OCR [27]. Grouping of events by their relative similarities and differences also helps to track events across time.…”

Section: Related Workmentioning

confidence: 99%

The use of topic evolution to help users browse and find answers in news video corpus

Neo

Ran

Goh

et al. 2007

Proceedings of the 15th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Earlier research in news video has been focusing mainly on improving retrieval accuracies given the limited amount of extractable video semantics. In this paper, we propose an enhancement to news video searching by leveraging extractable video semantics coupled with relevant external information resources to support event-based analysis; leading to discovery of topic hierarchy for browsing key events and supporting question answering (QA). We introduce topic browsing based on news structures obtained through hierarchical clustering and threading, with emphasis on interesting events determined by measuring the amount of "web activities" on these events on Blog sites. For QA, we employ extensive query analysis to obtain various query features in addition to the topic hierarchical structures to answer both context-oriented and visual-oriented questions. Our main contributions includes: (a) combining multimodal event information extracted from news video, web news articles and news blogs to support event analysis, (b) introducing topic evolution browsing based on users' interest and (c) extending QA on top of topic hierarchy to handle various types of specialized video queries. Experiments performed on 70 hours of multilingual news from TRECVID 2005 dataset shows that the proposed approach is effective and appealing to users.

show abstract

Complementary video and audio analysis for broadcast news archives

Cited by 26 publications

References 6 publications

Automatic Summarization

Automatic Summarization

Multimodal signal processing for meetings: an introduction

The use of topic evolution to help users browse and find answers in news video corpus

Contact Info

Product

Resources

About