Balancing thread based navigation for targeted video search

Abstract

Various query methods for video search exist. Because of the semantic gap each method has its limitations. We argue that for effective retrieval query methods need to be combined at retrieval time. However, switching query methods often involves a change in query and browsing interface, which puts a heavy burden on the user. In this paper, we propose a novel method for fast and effective search trough large video collections by embedding multiple query methods into a single browsing environment. To that end we introduced the notion of query threads, which contain a shot-based ranking of the video collection according to some feature-based similarity measure. On top of these threads we define several thread-based visualizations, ranging from fast targeted search to very broad exploratory search, with the ForkBrowser as the balance between fast search and video space exploration. We compare the effectiveness and efficiency of the ForkBrowser with the CrossBrowser on the TRECVID 2007 interactive search task. Results show that different query methods are needed for different types of search topics, and that the ForkBrowser requires significantly less user interactions to achieve the same result as the CrossBrowser. In addition, both browsers rank among the best interactive retrieval systems currently available.

The Video Browser Showdown: a live evaluation of interactive video search tools

“…Many browsing tools, with better interaction means than provided by a typical video player, have been presented in the literature (for a detailed review see [29]). While many of them are advanced navigation methods (e.g., [7,8]), extended video players (e.g., [10,14,18]) or enhanced video content visualizations [6], some are highly sophisticated browsing tools (e.g., [1,25,30]). These sophisticated tools provide very specific interfaces and advanced interaction methods, such as combined mouse/keyboard interaction for 3D navigation, table-of-content navigation in videos, navigation trees and spatial interaction (e.g., [12,13,22,23,26,28]).…”

Symbiosis between the TRECVid benchmark and video libraries at the Netherlands Institute for Sound and Vision

“…Searchers (experts and non-experts) will use more than text queries if available: concepts, visual similarity, temporal browsing, positive and negative relevance feedback. This can be seen clearly in the activities of the VideOlympics 4 and also in work by Christel [2] and by de Rooij et al [4].…”

“…This is confirmed by a recent user study [30] involving a number of broadcast professionals; -the feasibility of automatic tagging based on analysis of a shot's visual and aural content, not just on file names or manually assigned tags, even on large amounts of video of interest, is now available to us [23]; -there is a varying but often significant usefulness of text from speech as a basis for video retrieval [25]; -there is significant impact of the human in the semiautomated search process or in the video tagging loop [25]; -there is a feasibility and acceptance of search (interfaces) driven by more than just keyword input but rather also by content-based approaches such as visual concepts, visual similarity, temporal browsing, positive and negative feedback, etc., presented in a variety of designs [4]; -there is an increase in performance of automatic tagging systems using more than one keyframe per shot to represent the shot and the concomitant need for faster processing [21].…”

User variance and its impact on video retrieval benchmarking

“…For example, MediaMill at the University of Amsterdam build rich visualizations of the resultspace (e.g. the Fork-, Cross-and Rotor-Browser) that enable users to easily explore the full depth of often-complex result-sets [6]. The team from the National University of Singapore pushes the boundaries of 'extreme retrieval' by forcing the user to make judgments on a result's relevance within a very limited time window [13].…”