Wavelet-based video indexing and querying

Abstract

We present several algorithms suitable for analysis of broadcast video. First, we show how wavelet analysis of frames of video can be used to detect transitions between shots in a video stream, thereby dividing the stream into segments. Next we describe how each segment can be inserted into a video database using an indexing scheme that involves a wavelet-based "signature." Finally, we show that during a subsequent broadcast of a similar or identical video clip, the segment can be found in the database by quickly searching for the relevant signature. The method is robust against noise and typical variations in the video stream, even global changes in brightness that can fool histogram-based techniques. In the paper, we compare experimentally our shot transition mechanism to a color histogram implementation, and also evaluate the effectiveness of our databasesearching scheme. Our algorithms are very efficient and run in realtime on a desktop computer. We describe how this technology could be employed to construct a "smart VCR" that was capable of alerting the viewer to the beginning of a specific program or identifying commercials and then muting the volume on the TV.

A framework for aligning and indexing movies with their script

“…We implemented and used an algorithm described by Salesin et al for shot change detection using Haar wavelet coefficients [8]. The algorithm computes a distance between successive frames, and uses thresholds to detect candidate shot cuts.…”

2C-Net: integrate image compression and classification via deep neural network

“…Seeking effective representations in a shared feature space is another promising way to avoid image reconstruction for intelligent vision tasks [43][44][45]. Many early researches attempted to execute color histograms [3] and texture features [4] from Discrete Cosine Transform (DCT) domain or wavelet domain for fast image retrieval.…”

A robust multi-feature cut detection algorithm for video segmentation

“…A frame difference measure is defined on the joint probability images considering only values near the diagonal of each image. In [19] the Haar wavelet transform is used to compute a wavelet signature from each frame. The signature is constructed considering the most significant wavelet coefficients.…”