Generalized statistics framework for rate distortion theory

Venkatesan, R. C.; Plastino, A.

doi:10.1016/j.physa.2009.02.003

Cited by 11 publications

(42 citation statements)

References 49 publications

(125 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This suggests possible connections between coding theory and the measure of complexity in nonextensive statistical mechanics. Related works are the study of generalized channel capacities [9], the notion of nonadditive information content [10], the presentation of a generalized rate distorsion theory [11]. The first section is devoted to a very short presentation of the source coding context, and to the presentation of the fundamental Shannon source coding theorem.…”

Section: Introductionmentioning

confidence: 99%

Source coding with escort distributions and Rényi entropy bounds

Bercher

2009

Physics Letters A

View full text Add to dashboard Cite

We discuss the interest of escort distributions and Rényi entropy in the context of source coding. We first recall a source coding theorem by Campbell relating a generalized measure of length to the Rényi-Tsallis entropy. We show that the associated optimal codes can be obtained using considerations on escort-distributions. We propose a new family of measure of length involving escort-distributions and we show that these generalized lengths are also bounded below by the Rényi entropy. Furthermore, we obtain that the standard Shannon codes lengths are optimum for the new generalized lengths measures, whatever the entropic index. Finally, we show that there exists in this setting an interplay between standard and escort distributions.

show abstract

Section: Introductionmentioning

confidence: 99%

Source coding with escort distributions and Rényi entropy bounds

Bercher

2009

Physics Letters A

View full text Add to dashboard Cite

show abstract

“…Instead, as established in this paper, the dual generalized K-Ld is a scaled Bregman divergence. Future work uses the results derived herein to analyze: (i) the generalized statistics rate distortion theory [2], (ii) the generalized statistics information bottleneck method [3] within the context of scaled Bregman divergences and scaled Bregman informations, and (iii) deformed statistics extensions of the minimum Bregman information principle and their applications in machine learning [36].…”

Section: Summary and Discussionmentioning

confidence: 99%

“…• (i) the generalized K-Ld defined by (6) subjected to the additive duality (dual generalized K-Ld (8) and (15)) is shown to be consistent with the canonical probability that maximizes the dual Tsallis entropy of the form [2,26]:…”

Section: Goal Of This Papermentioning

confidence: 98%

See 1 more Smart Citation

Scaled Bregman divergences in a Tsallis scenario

Venkatesan¹,

Plastino²

2011

Physica A: Statistical Mechanics and its Applications

Self Cite

View full text Add to dashboard Cite

There exist two different versions of the Kullback-Leibler divergence (K-Ld) in Tsallis statistics, namely the usual generalized K-Ld and the generalized Bregman K-Ld. Problems have been encountered in trying to reconcile them. A condition for consistency between these two generalized K-Ld-forms by recourse to the additive duality of Tsallis statistics is derived. It is also shown that the usual generalized K-Ld subjected to this additive duality, known as the dual generalized K-Ld, is a scaled Bregman divergence. This leads to an interesting conclusion: the dual generalized mutual information is a scaled Bregman information. The utility and implications of these results are discussed.

show abstract

“…The qualitative enhancement of the generalized IB method vis-á-vis an equivalent B-G-S model are demonstrated by the relevance-compression curves. A generalized Bregman RD (GBRD) model is presented [2]. A Tsallis-Bregman lower bound for the RD function is derived.…”

mentioning

confidence: 99%