Measuring time-frequency information content using the Renyi entropies

Abstract

Abstract-The generalized entropies of Rényi inspire new measures for estimating signal information and complexity in the time-frequency plane. When applied to a time-frequency representation (TFR) from Cohen's class or the affine class, the Rényi entropies conform closely to the notion of complexity that we use when visually inspecting time-frequency images. These measures possess several additional interesting and useful properties, such as accounting and cross-component and transformation invariances, that make them natural for time-frequency analysis. This paper comprises a detailed study of the properties and several potential applications of the Rényi entropies, with emphasis on the mathematical foundations for quadratic TFRs. In particular, for the Wigner distribution, we establish that there exist signals for which the measures are not well defined.

On classes of non-Gaussian asymptotic minimizers in entropic uncertainty principles

“…When λ tends to 1, by l'Hospital's rule, H λ converges to the Shannon entropy that will thus be denoted by continuity H 1 = H. The Rényi entropy is widely used, not only in physics (e.g. statistical mechanics, physics of turbulence, cosmology, see [10,11,12] and references therein), but in various other areas such as in signal processing (time scale analysis, decision problems, machine learning, see [4,13,14,15] and references therein), or image processing (image matching, image registration see [16,17] and references therein). …”

Waveform information from quantum mechanical entropy

“…Existing approaches generally rely on some secondary distribution or other probabilistic construct to which a measure of indeterminacy may be attributed. See for instance [2][3][4]. Although a variety of such well-formulated methods have been introduced, all feature some non-trivial dependence on model or parametrization.…”

An Entropy Based Method for Local Time-Adaptation of the Spectrogram

“…Some examples of this approach can be found in the literature: the idea of gathering a sparsity measure from Rényi entropies is detailed in [1], and in [14] a local time-frequency adaptive framework is presented exploiting this concept, even if no methods for perfect reconstruction are provided. In [21] sparsity is obtained through a regression model; a recent development in this sense is contained in [15] where a class of methods for analysis adaptation are obtained separately in the time and frequency dimension together with perfect reconstruction formulas: indeed no strategies for automatization are employed, and adaptation has to be managed by the user.…”

“…We consider measures borrowed from Information Theory and Probability Theory according to the interpretation of the analysis within a frame as a probability density [5]: our model is based on a class of entropy measures known as Rényi entropies which extend the classical Shannon entropy. The fundamental idea is that minimizing the complexity or information over a set of time-frequency representations of the same signal is equivalent to maximizing the concentration and peakiness of the analysis, thus selecting the best resolution tradeoff [1]: in the section Rényi Entropy of Spectrograms we describe how a sparsity measure can consequently be defined through an information measure. Finally, in the fourth section we provide a description of our algorithm and examples of adapted spectrogram for different sounds.…”

“…Thanks to this interpretation, some techniques belonging to the domain of Probability and Information Theory can be applied to our problem: in particular, the concept of entropy can be extended to give a sparsity measure of a time-frequency density. A promising approach [1] takes into account Rényi entropies, a generalization of the Shannon entropy: the application to our problem is related to the concept that minimizing the complexity or information of a set of time-frequency representations of a same signal is equivalent to maximizing the concentration, peakiness, and therefore the sparsity of the analysis. Thus we will consider as best analysis the sparsest one, according to the minimal entropy evaluation.…”