Learning query-class dependent weights in automatic video retrieval

Yan, Rong; Yang, Jun; Hauptmann, Alexander G.

doi:10.1145/1027527.1027661

Cited by 108 publications

(97 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another research trend emphasizes on the refinement of the search results based on the user selected results, usually referred to as the "relevance feedback" [10][1], where the search queries are refined by finding the prominent features and the queries are expanded based on the concept ontology. Given the training data, the query that best fits to the target topic can be estimated without the relevance feedback process, such as the mixture of experts [11] and the query-dependent search [6].…”

Section: Introductionmentioning

confidence: 99%

Automatic Query Expansion for News Video Retrieval

Zhai

Liu

Shah

2006

2006 IEEE International Conference on Multimedia and Expo

View full text Add to dashboard Cite

In this paper, we present an integrated system for news video retrieval. The proposed system incorporates both speech and visual information in the search mechanisms. The initial search is based on the automatic speech recognition (ASR) transcript of video. Based on the relevant shots selected from the initial search round, keyword histograms are automatically generated for the refinement of the search query, such that the reformulated query fits better to the target topic. We have also developed an image-based refinement module, which uses the region analysis of the video key-frames. SR-tree like indexing structure is constructed for the region features, and the imageto-image similarity is computed using the Earth Mover's Distance. By performing a series of relevance feedback processes, the set of the true relevant shots is expanded significantly. The proposed system has been applied to a large open-benchmark news video dataset, and very satisfactory improvements have been obtained by applying the proposed automatic query expansion and the region-based refinement.

show abstract

Section: Introductionmentioning

confidence: 99%

Automatic Query Expansion for News Video Retrieval

Zhai

Liu

Shah

2006

2006 IEEE International Conference on Multimedia and Expo

View full text Add to dashboard Cite

show abstract

“…Query-Class-Dependent Models: To improve upon these query-independent systems, Chua et al [7] and Yan et al [17] independently proposed query-class-dependent approaches, which relied upon human-defined classes of queries. Chua [ The general frameworks of both proposed systems follow similar strategies, beyond the selection of the query classes.…”

Section: B Multimodal Search Over Image and Video Collectionsmentioning

confidence: 99%

“…Kang and Kim [5], on the other hand, build language models for the various types of documents that are relevant to the classes of queries in their system and classify incoming queries based on their likelihood of being generated by each of the language models. The trend in much of the recent work in multimodal video search has been to perform some lightweight natural language processing on the incoming query text to extract counts of various parts of speech as well as named entities along with the appearance of predefined lists of keywords related to specific topics [3], [7], [14], [17]. This proposal, which is heavily reliant on reasonable part-of-speech and named entity detection, is somewhat of an artifact of the TRECVID test domain, where the provided text queries are complete sentences, which is generally not the rule in many consumer applications.…”

Section: A Understanding Search Intentmentioning

confidence: 99%

“…Many research efforts have proposed various methods for adapting retrieval strategies according to the query provided by the user in a number of search applications. One such strategy is to predefine a set of query Bclasses,[ which are sets of queries in which the optimal search strategies are similar for all queries contained within the class [3]- [5], [7], [11], [13], [14], [17]. Incoming queries can be classified into one of the classes based on some light natural language analysis of the query.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Query-Adaptive Fusion for Multimodal Search

2008

View full text Add to dashboard Cite

| We conduct a broad survey of query-adaptive search strategies in a variety of application domains, where the internal retrieval mechanisms used for search are adapted in response to the anticipated needs for each individual query experienced by the system. While these query-adaptive approaches can range from meta-search over text collections to multimodal search over video databases, we propose that all such systems can be framed and discussed in the context of a single, unified framework. In our paper, we keep an eye towards the domain of video search, where search cues are available from a rich set of modalities, including textual speech transcripts, low-level visual features, and high-level semantic concept detectors. The relative efficacy of each of the modalities is highly variant between many types of queries. We observe that the state of the art in query-adaptive retrieval frameworks for video collections is highly dependent upon the definition of classes of queries, which are groups of queries that share similar optimal search strategies, while many applications in text and web retrieval have included many advanced strategies, such as direct prediction of search method performance and inclusion of contextual cues from the searcher. We conclude that such advanced strategies previously developed for text retrieval have a broad range of possible applications in future research in multimodal video search.

show abstract

“…Some prior work exists on classifying topics and treating each topic differently [4,5], there the classification is used to decide how to combine the various retrieval modalities (e.g., ASR, visual). An approach to use a surface feature, the time within the video, for searching for topics classified as weather news or sports event is discussed in [6].…”

Section: Topic Classificationmentioning

confidence: 99%

Surface Features in Video Retrieval

Westerveld

Vries

Ramirez

2006

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. This paper assesses the usefulness of surface features in a multimedia retrieval setting. Surface features describe the metadata or structure of a document rather than the content. We note that the distribution of these features varies across topics. The paper shows how these distributions can be obtained through relevance feedback and how this allows for adaptation of (content-based) search results for topic or user preference. An analysis of the distribution of surface features in the TRECVID collection indicates that they are potentially useful, and a preliminary feedback experiment confirms that exploiting surface features can improve retrieval effectiveness.

show abstract

Learning query-class dependent weights in automatic video retrieval

Cited by 108 publications

References 15 publications

Automatic Query Expansion for News Video Retrieval

Automatic Query Expansion for News Video Retrieval

Query-Adaptive Fusion for Multimodal Search

Surface Features in Video Retrieval

Contact Info

Product

Resources

About