Majorization, Csiszár divergence and Zipf-Mandelbrot law

Abstract

In this paper we show how the Shannon entropy is connected to the theory of majorization. They are both linked to the measure of disorder in a system. However, the theory of majorization usually gives stronger criteria than the entropic inequalities. We give some generalized results for majorization inequality using Csiszár f-divergence. This divergence, applied to some special convex functions, reduces the results for majorization inequality in the form of Shannon entropy and the Kullback-Leibler divergence. We give several applications by using the Zipf-Mandelbrot law.

Bounds for Csiszár divergence and hybrid Zipf‐Mandelbrot entropy

“…For more results related Zipf-Mandelbrot entropies, see previous studies. [8][9][10][11][12][13][14][15] Further generalization of Zipf-Mandelbrot entropy is hybrid Zipf-Mandelbrot entropy, 16 which is given by…”

Bounded Rational Decision-Making from Elementary Computations That Reduce Uncertainty

“…Even though a large amount of research has been done on majorization theory, from the early works [ 29 , 34 , 38 ] through further developments [ 27 , 30 , 31 , 32 , 36 , 77 , 78 ] to modern applications [ 39 , 40 , 41 ], there is a lack of results on the more general concept of relative majorization. This does not seem to be due to a lack of interest, as can be seen from the results [ 28 , 57 , 58 , 79 ], but mostly because relative majorization looses some of the appealing properties of majorization which makes it harder to deal with, for example that permutations no longer leave the ordering invariant, in contrast to the case of a uniform prior. This restriction does, however, not affect our application of the concept to decision-making, as permutations are not considered as elementary computations, since they do not diminish uncertainty.…”

Shannon type inequalities via time scales theory

“…In recent times, Shannon entropy and Zipf-Mandelbrot law have been the topics of great interest, see for example [1,9,11,12]. The concept of Shannon entropy, the central source of information theory, is sometimes referred to as measure of uncertainty.…”