The use of topic evolution to help users browse and find answers in news video corpus

Abstract

Earlier research in news video has been focusing mainly on improving retrieval accuracies given the limited amount of extractable video semantics. In this paper, we propose an enhancement to news video searching by leveraging extractable video semantics coupled with relevant external information resources to support event-based analysis; leading to discovery of topic hierarchy for browsing key events and supporting question answering (QA). We introduce topic browsing based on news structures obtained through hierarchical clustering and threading, with emphasis on interesting events determined by measuring the amount of "web activities" on these events on Blog sites. For QA, we employ extensive query analysis to obtain various query features in addition to the topic hierarchical structures to answer both context-oriented and visual-oriented questions. Our main contributions includes: (a) combining multimodal event information extracted from news video, web news articles and news blogs to support event analysis, (b) introducing topic evolution browsing based on users' interest and (c) extending QA on top of topic hierarchy to handle various types of specialized video queries. Experiments performed on 70 hours of multilingual news from TRECVID 2005 dataset shows that the proposed approach is effective and appealing to users.

Exploiting story-level context to improve video search

“…The system in [4] is among the earlier pioneers in exploiting external text resources such as a parallel news corpus. The main idea is to tap on news sources that were reporting around the broadcast time of the video, with the hope of uncovering useful complementary information to help video analysis.…”

“…We use a simple HTML parser to extract the text from the resulting return articles. It is noteworthy that compared to [4,5] where thousands of articles are used, we only capture the top-100 articles. Let's call this corpus E. All news articles in E are stopped and stemmed.…”

“…While providing a sufficient basis for visual-based retrieval, algorithms that attempt to harness the semantic power in the accompanying close caption text or automatic speech transcripts are often faced with the temporal mismatch problem --the object and/or scene that is being mentioned may not necessarily appear in the video concurrently [2]. To cope with this, coupled with advances in news story segmentation [3], a general strategy shown in recent state-of-art systems is to combine (text [4], visual features [5]) information in the neighboring shots that make up the news story. The main idea is that because the news story unit is semantically coherent, we can expect a strong relatedness in all visual shots and speech-text.…”

Web video topics discovery and structuralization with social network

“…In the text domain, Chen et al [6] used aging theory to model the life-cycles of a topic. In the video domain, Neo et al modeled a topic as a ranked list of videos according to the time line [13], and each video is further attached with its description and relevant articles collected from multiple resources. In [26], Wu et al adopted a binary tree to model and thread the relationship of events in news topics.…”

Beyond search

“…In this scenario, Yang et al [2003] extended the techniques from text processing to video question answering by modeling the Web linguistic knowledge. Neo et al [2007] enhanced news video searching by leveraging extractable video semantics coupled with relevant external information resources to support eventbased analysis. In Zhu et al [2003], a hierarchical video content description and summarization strategy supported by a joint textual and visual similarity is presented.…”