Tsallis entropy-based information measures for shot boundary detection and keyframe selection

Abstract

Automatic shot boundary detection and keyframe selection constitute major goals in video processing. We propose two different information-theoretic approaches to detect the abrupt shot boundaries of a video sequence. These approaches are, respectively, based on two information measures, Tsallis mutual information and Jensen-Tsallis divergence, that are used to quantify the similarity between two frames. Both measures are also used to find out the most representative keyframe of each shot. The representativeness of a frame is basically given by its average similarity with respect to the other frames of the shot. Several experiments analyze the behavior of the proposed measures for different color spaces (RGB, HSV, and Lab), regular binnings, and entropic indices. In particular, the Tsallis mutual information for the HSV and Lab color spaces with only 8 regular bins for each color component and an entropic index between 1. 5 and 1. 8 substantially improve the performance of previously proposed methods based on mutual information and Jensen-Shannon divergenceThis work has been funded in part by grants from the Spanish Government (Nr. TIN2010-21089-C03-01), from the Catalan Government (Nr. 2009-SGR-643 and Nr. 2010-CONE2-00053), and from the Natural Science Foundation of China (61179067, 61103005, 60879003

Information Theory Tools for Image Processing

“…From the initial work of Cernekova et al [31] new algorithms have been developed [178,186,187]. From the initial work of Cernekova et al [31] new algorithms have been developed [178,186,187].…”

“…In this section, we present two approaches based, respectively, on Tsallis mutual information and Jensen-Tsallis divergence to detect the abrupt shot boundaries of a video sequence [178]. en, we describe three new measures to extract the most representative keyframes.…”

“…Cernekova et al's work [31] using Shannon MI between frames has been extended by using Tsallis entropy and several color spaces [178]. Specifically, the informational frame similarity is defined by…”

A multimedia system to produce and deliver video fragments on demand on parliamentary websites

“…The shot boundary detection problem has traditionally involved the computation of visual differences between consecutive frames and a shot has been detected when this difference has been higher than a certain threshold [31,43,46]. In our approach, the feature used in each frame to detect the shot transition is the color, which has proven to be effective [47]. Thus, each frame framei is modeled by its color probability distribution Pi, and the shot boundary detection problem is stated as the search for a significant difference in color distribution between consecutive frames, framei and framei+1.…”

Generalized Wavelet Fisher’s Information of1/fαSignals

“…Moreover, Fisher information in wavelet domain was used in [12] to detect weak structural breaks in 1/ signals. Tsallis entropies, on the other hand, have been used for studying a variety of signal 2 Advances in Mathematical Physics characteristics [13] and its wavelet counterparts are currently being used from structural damage identification in [14], signal classification in [15], and for detecting structural breaks in the mean in pure-power law (PPL) signals [16] among others [17,18]. In general, with the use of wavelet based information tools, significant improvements can be achieved in the overall analysis and estimation of 1/ signals.…”