Comparing segmentation strategies for efficient video passage retrieval

Abstract

We compare the effect of different text segmentation strategies on speech based passage retrieval of video. Passage retrieval has mainly been studied to improve document retrieval and to enable question answering. In these domains best results were obtained using passages defined by the paragraph structure of the source documents or by using arbitrary overlapping passages. For the retrieval of relevant passages in a video, using speech transcripts, no author defined segmentation is available. We compare retrieval results from 4 different types of segments based on the speech channel of the video: fixed length segments, a sliding window, semantically coherent segments and prosodic segments. We evaluated the methods on the corpus of the MediaEval 2011 Rich Speech Retrieval task. Our main conclusion is that the retrieval results highly depend on the right choice for the segment length. However, results using the segmentation into semantically coherent parts depend much less on the segment length. Especially, the quality of fixed length and sliding window segmentation drops fast when the segment length increases, while quality of the semantically coherent segments is much more stable. Thus, if coherent segments are defined, longer segments can be used and consequently less segments have to be considered at retrieval time.

Experiments with Segmentation Strategies for Passage Retrieval in Audio-Visual Documents

“…Surprisingly, in text retrieval, the majority of authors (e.g., [31,3,13,28]) show that the segmentation using sliding windows and creating overlapping segments of a regular length is the most successful approach to segmentation and its subsequent usage in IR. It is also demonstrated that this approach is sensitive to the window length which needs to be tuned on training data.…”

“…Segmentation for audio-visual recordings has not been widely studied; some experiments were performed by Eskevich et al [7] and Wartena [31]. The Story Segmentation Task of TRECVID 2003 [24] and 2004 [25] focused on identification of story boundaries in video recordings, however, the detected boundaries were not subsequently used for IR.…”

Segmentation strategies for passage retrieval in audio-visual documents

“…[3,5]). Our experiments performed on the test collection used in the Search subtask of the Search and Hyperlinking Task in MediaEval Benchmarking 2012 1 confirm those findings and show that parameters (segment length and shift) tuning for a specific test collection can further improve the results.…”

Comparing retrieval effectiveness of alternative content segmentation methods for Internet video search

“…We focus on the behaviour of the vocabulary of individual queries and content of the segment units. A more quantitative study analyzing different aspects of the text and the segments can be found in [17].…”

“…For the segmentation stage, fragments were defined as a sequence of sentences of n non-stop-words. In this investigation n = 20, 40 were used, more on n variations in [17]. Sentences were derived on the basis of punctuation (fullstop = sentence end) hypothesized by the ASR system and included in the transcript.…”