Scaled Bregman divergences in a Tsallis scenario

Abstract

There exist two different versions of the Kullback-Leibler divergence (K-Ld) in Tsallis statistics, namely the usual generalized K-Ld and the generalized Bregman K-Ld. Problems have been encountered in trying to reconcile them. A condition for consistency between these two generalized K-Ld-forms by recourse to the additive duality of Tsallis statistics is derived. It is also shown that the usual generalized K-Ld subjected to this additive duality, known as the dual generalized K-Ld, is a scaled Bregman divergence. This leads to an interesting conclusion: the dual generalized mutual information is a scaled Bregman information. The utility and implications of these results are discussed.

Deformed statistics Kullback–Leibler divergence minimization within a scaled Bregman framework

“…The extension of the generalization of the results derived in Ref. [14] is presented in Sections 4 and 5 of this Letter.…”

“…Note that for mutual information based models, defining the scaled Bregman information as the normal averages expectation of the dual generalized K-Ld [14], the Pythagorean theorem derived for the dual generalized K-Ld in this Letter provides the foundation to extend the optimality of minimum Bregman information principle [12], [32] which has immense utility in machine learning and allied disciplines, and, the Bregman projection theorem to the case of deformed statistics. Finally, the Pythagorean theorem and the minimum dual generalized K-Ld principle developed in this Letter serve as a basis to generalize the concept of I-projections [33] to the case of deformed statistics.…”

“…In Ref. [14], it was shown that the dual generalized K-Ld is a scaled Bregman divergence. This was demonstrated in a discrete setting.…”

“…A recent study [14] has established that the dual generalized K-Ld is a scaled Bregman divergence in a discrete setting. Further, Ref.…”

Invertible mappings and the large deviation theory for the $q$-maximum entropy principle

“…It is noteworthy to mention that the additive duality was recently employed to successfully demonstrate that the dual generalized Kullback-Leibler divergence is a scaled Bregman divergence [24,25]. This paper derives the necessary conditions to reconcile the dual Tsallis maximum entropy principle with the asymptotic frequencies obtained from large deviation theory (i.e.…”