A Comparison of Score, Rank and Probability-Based Fusion Methods for Video Shot Retrieval

Abstract

It is now accepted that the most eective video shot retrieval is based on indexing and retrieving clips using multiple, parallel modalities such as text-matching, image-matching and feature matching and then combining or fusing these parallel retrieval streams in some way. In this paper we investigate a range of fusion methods for combining based on multiple visual features (colour, edge and texture), for combining based on multiple visual examples in the query and for combining multiple modalities (text and visual). Using three TRECVid collections and the TRECVid search task, we specically compare fusion methods based on normalised score and rank that use either the average, weighted average or maximum of retrieval results from a discrete Jelinek-Mercer smoothed language model. We also compare these results with a simple probability-based combination of the language model results that assumes all features and visual examples are fully independent.

Beyond Shot Retrieval: Searching for Broadcast News Items Using Language Models of Concepts

“…As a result we focus on working with concepts extracted from the content of images. Current concept based video retrieval systems normally operate on a fixed-number of features per retrieval unit, for example the confidence scores of detectors for a number of concepts [6,18]. Therefore, it is difficult to extend these models to search for news items of varying length.…”

QDFA: Query-Dependent Feature Aggregation for Medical Image Retrieval

“…The distance between two CLD vectors was calculated as To evaluate the retrieval performance of QDFA, CombSumScore [4], which has the best performance in Deselaers' feature aggregation functions, LSVMC [2] and OSVM-QDFF [5] were implemented as references. We used the LIBSVM [8] to solve SVMs (for LSVMC), OSVMs (for OSVM-QDFF) and FSVMs (for QDFA).…”

“…We focus on developing a new feature aggregation method in this letter. Deselaers et al [4] applied CombSumScore, CombMaxScore, CombSumRank, CombMaxRank functions to aggregate multiple similarities for multi-feature and multi-example queries. Deselaers' methods are queryindependent, which apply the same feature aggregation function for different queries, without considering that a special feature is not equally important for different queries.…”

Text based approaches for content-based image retrieval on large image collections

“…It would be interesting to compare our results to that of more contemporary ranking techniques (and fusion models) such as [11] to see how this approach compares. To do this though would require the current approach to be extended to incorporate the other MPEG-7 features that we regularly make use of, including Colour Layout and Homogenous Texture.…”