Sharp bounds on Arimoto's conditional Rényi entropies between two distinct orders

Abstract

This study examines sharp bounds on Arimoto's conditional Rényi entropy of order β with a fixed another one of distinct order α = β. Arimoto inspired the relation between the Rényi entropy and the r -norm of probability distributions, and he introduced a conditional version of the Rényi entropy. From this perspective, we analyze the r -norms of particular distributions. As results, we identify specific probability distributions whose achieve our sharp bounds on the conditional Rényi entropy. The sharp bounds derived in this study can be applicable to other information measures, e.g., the minimum average probability of error, the Bhattacharyya parameter, Gallager's reliability function E0, and Sibson's α-mutual information, whose are strictly monotone functions of the conditional Rényi entropy.

A Direct Link between Rényi–Tsallis Entropy and Hölder’s Inequality—Yet Another Proof of Rényi–Tsallis Entropy Maximization

“…The idea of the proof is different from those offered in previous studies (for instance, [23][24][25][26][27]) as explained below. Interestingly, the technique developed in this study might possibly be useful for tackling more complicated problems regarding optimization issues in information theory and other research areas, such as the conditional Rényi entropy (as in [28][29][30]), for instance.…”

Generalizations of Fano’s Inequality for Conditional Information Measures via Majorization Theory

“…We show that our Fano-type inequalities can be specialized to some known generalizations of Fano’s inequality [ 20 , 21 , 22 , 23 ] on Shannon’s and Rényi’s information measures. Therefore, one of our technical contributions is a unified proof of Fano’s inequality for conditional information measures via majorization theory.…”

“…In [ 54 ], Ben-Bassat–Raviv explored several inequalities between the (unconditional) Rényi entropy and the error probability. Generalizations of Fano’s inequality from the conditional Shannon entropy to Arimoto’s conditional Rényi entropy introduced in [ 8 ] were recently and independently investigated by Sakai–Iwata [ 22 ] and Sason–Verdú [ 23 ]. Specifically, Sakai–Iwata [ 22 ] provided sharp upper/lower bounds on for fixed with two distinct orders .…”

“…Generalizations of Fano’s inequality from the conditional Shannon entropy to Arimoto’s conditional Rényi entropy introduced in [ 8 ] were recently and independently investigated by Sakai–Iwata [ 22 ] and Sason–Verdú [ 23 ]. Specifically, Sakai–Iwata [ 22 ] provided sharp upper/lower bounds on for fixed with two distinct orders . In other words, they gave explicit formulas of the following minimization and maximization, respectively.…”

“…As is a strictly monotone function of the minimum average probability of error if , both functions and can be thought of as reverse and forward Fano inequalities on , respectively (cf. Section V in the arXiv paper [ 22 ]). Sason–Verdú [ 23 ] also gave generalizations of the forward and reverse Fano’s inequalities on .…”