A characterization of the Shannon ordering of communication channels

Abstract

The ordering of communication channels was first introduced by Shannon. In this paper, we aim to find a characterization of the Shannon ordering. We show that W ′ contains W if and only if W is the skew-composition of W ′ with a convex-product channel. This fact is used to derive a characterization of the Shannon ordering that is similar to the Blackwell-Sherman-Stein theorem. Two channels are said to be Shannon-equivalent if each one is contained in the other. We investigate the topologies that can be constructed on the space of Shannon-equivalent channels. We introduce the strong topology and the BRM metric on this space. Finally, we study the continuity of a few channel parameters and operations under the strong topology.

Comparison of noisy channels and reverse data-processing theorems

“…This is similar to the degradability condition, in the sense that it is based on the notion of simulability (of one channel by means of another), however, the set of transformations allowed is much more general, as it includes general encodings, decodings, and free use of shared randomness. Channel inclusion too can be studied from the general viewpoint of statistical comparison: this has been done in the classical scenario in [27], [28], whereas, in the quantum scenario, a similar ordering, in which not only shared randomness but also forward classical communication is freely available, has been studied in [29].…”

Semantic information, autonomous agency and non-equilibrium statistical physics

“…In order to make things rigorous, we require a formal definition of how to transform an actual distribution into an intervened distribution. While we do not claim that there is a single best choice for defining interventions, we propose to use information-theoretic "coarse-graining" methods to scramble the channel between the system and environment [74][75][76][77][78][79].…”

“…In order to make things rigorous, we require a formal definition of how to transform an actual distribution into an intervened distribution. While we do not claim that there is a single best choice for defining interventions, we propose to use information-theoretic 'coarse-graining' methods to scramble the channel between the system and environment [74][75][76][77][78][79]. Importantly, such methods allow us to choose different coarse-grainings, which lets us vary the syntactic information that is preserved under different interventions, and the resulting viability of the system at time t. By considering different interventions, we define a trade-off between the amount of preserved syntactic information versus the resulting viability of the system at time t. This trade-off is formally represented by an information/viability curve (figure 1c), which is loosely analogous to the rate-distortion curves in information theory [36].…”

“…X0jY0 is sometimes called a 'Markov approximation' of the actual channel p X 0 jY 0 [77], which is itself a special case of the so-called channel pregarbling or channel input-degradation [77][78][79]. Pre-garbling is a principled way to destroy part of the information flowing across a channel, and has important operationalizations in terms of coding and game theory [78].…”

Topological Structures on DMC Spaces †

“…The Shannon deficiency that was introduced in [ 17 ] compares a particular channel with the Shannon-equivalence-class of another channel, but it is not a metric distance between Shannon-equivalence-classes. In [ 18 ], we provide a characterization of the Shannon ordering and we prove that some of the results of this paper holds for the space of Shannon-equivalent channels.…”