Source coding with escort distributions and Rényi entropy bounds

Bercher, Jean‐François

doi:10.1016/j.physleta.2009.07.015

Cited by 48 publications

(52 citation statements)

References 17 publications

(36 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the future, we plan to consider a more general family of entropy functions, including Rényi and Tsallis entropies, which are of great importance in the theory of coding and related problems [6,20,21]. Moreover, there also arises a natural question concerning the compression of n-tuple random variables.…”

Section: Discussionmentioning

confidence: 99%

Entropy Approximation in Lossy Source Coding Problem

Śmieja

Tabor

2015

Entropy

View full text Add to dashboard Cite

In this paper, we investigate a lossy source coding problem, where an upper limit on the permitted distortion is defined for every dataset element. It can be seen as an alternative approach to rate distortion theory where a bound on the allowed average error is specified. In order to find the entropy, which gives a statistical length of source code compatible with a fixed distortion bound, a corresponding optimization problem has to be solved. First, we show how to simplify this general optimization by reducing the number of coding partitions, which are irrelevant for the entropy calculation. In our main result, we present a fast and feasible for implementation greedy algorithm, which allows one to approximate the entropy within an additive error term of log 2 e. The proof is based on the minimum entropy set cover problem, for which a similar bound was obtained.

show abstract

Section: Discussionmentioning

confidence: 99%

Entropy Approximation in Lossy Source Coding Problem

Śmieja

Tabor

2015

Entropy

View full text Add to dashboard Cite

show abstract

“…We also have an interesting connection with source coding, which was first reported in [10]. The mains formulas and comparisons are given on Table 1.…”

Section: Source Codingmentioning

confidence: 95%

On escort distributions, q-gaussians and Fisher information

Mohammad‐Djafari¹,

Bercher

Bessìère³

2011

AIP Conference Proceedings

Self Cite

View full text Add to dashboard Cite

Abstract. Escort distributions are a simple one parameter deformation of an original distribution p. In Tsallis extended thermostatistics, the escort-averages, defined with respect to an escort distribution, have revealed useful in order to obtain analytical results and variational equations, with in particular the equilibrium distributions obtained as maxima of Rényi-Tsallis entropy subject to constraints in the form of a q-average. A central example is the q-gaussian, which is a generalization of the standard gaussian distribution.In this contribution, we show that escort distributions emerge naturally as a maximum entropy trade-off between the distribution p(x) and the uniform distribution. This setting may typically describe a phase transition between two states. But escort distributions also appear in the fields of multifractal analysis, quantization and coding with interesting consequences. For the problem of coding, we recall a source coding theorem by Campbell relating a generalized measure of length to the Rényi-Tsallis entropy and exhibit the links with escort distributions together with pratical implications.That q-gaussians arise from the maximization of Rényi-Tsallis entropy subject to a q-variance constraint is a known fact. We show here that the (squared) q-gaussian also appear as a minimum of Fisher information subject to the same q-variance constraint.

show abstract

“…Shannon [1] provided an operational meaning to his entropy through a source coding theorem by establishing the limits to possible data compression. Bercher [9] discussed the interest of escort distributions and Rényi entropy in the context of source coding whereas Parkash and Kakkar [10] developed new mean codeword lengths and proved source coding theorems. Huffman [11] introduced a procedure for designing a variable length source code which achieves performance close to Shannon's entropy bound.…”

Section: Introductionmentioning

confidence: 99%

An Algorithm to Generate Probabilities with Specified Entropy

Parkash¹,

Kakkar²

2015

View full text Add to dashboard Cite

The present communication offers a method to determine an unknown discrete probability distribution with specified Tsallis entropy close to uniform distribution. The relative error of the distribution obtained has been compared with the distribution obtained with the help of mathematica software. The applications of the proposed algorithm with respect to Tsallis source coding, Huffman coding and cross entropy optimization principles have been provided.

show abstract

Source coding with escort distributions and Rényi entropy bounds

Cited by 48 publications

References 17 publications

Entropy Approximation in Lossy Source Coding Problem

Entropy Approximation in Lossy Source Coding Problem

On escort distributions, q-gaussians and Fisher information

An Algorithm to Generate Probabilities with Specified Entropy

Contact Info

Product

Resources

About