Learning User Queries in Multimodal Dissimilarity Spaces

Abstract

Abstract. Different strategies to learn user semantic queries from dissimilarity representations of video audio-visual content are presented. When dealing with large corpora of videos documents, using a feature representation requires the online computation of distances between all documents and a query. Hence, a dissimilarity representation may be preferred because its offline computation speeds up the retrieval process. We show how distances related to visual and audio video features can directly be used to learn complex concepts from a set of positive and negative examples provided by the user. Based on the idea of dissimilarity spaces, we derive three algorithms to fuse modalities and therefore to enhance the precision of retrieval results. The evaluation of our technique is performed on artificial data and on the complete annotated TRECVID corpus.

Asymmetric Learning and Dissimilarity Spaces for Content-Based Retrieval

“…For the sake of evaluation, learning in QDS has been done through a simple linear-SVM. However, in order to build an effective multimedia retrieval system as the one we presented in [2], non-linear approaches and more sophisticated strategies may be enlisted to cope with real world non-linearly distributed multimodal documents.…”

“…In [2], we proposed a similarity-based representation that goes further the nearest-neighbour model by allowing non-linear mapping of the low-level distance measures to the high-level concept space. Based on Dissimilarity Spaces (DS) introduced by Pekalska et al [8], we have defined representation spaces adapted to the query-by-example paradigm.…”

Interactive Search by Direct Manipulation of Dissimilarity Space

“…Feature selection is a special case of feature weighting, where the weights of the eliminated features are set to 0. As the update changes only, (1) is rewritten as (2) In [33], the authors concentrate on exploring the distribution of the data set. A subspace of the feature space is found, and a quadratic similarity functions is learnt.…”

“…The selection of relevant images as prototypes makes the learning on dissimilarity space simpler [2] (see example on Fig. 2).…”

“…A similar approach has been applied in [2]. In this reference, the authors also consider dissimilarity space as a substitute for the feature space.…”