Approximation theory of output statistics

Abstract

Abstract-Given a channel and an input process, the minimum randomness of those input processes whose output statistics approximate the original output statistics with arbitrary accuracy is studied. The notion of resolvability of a channel, defined as the number of random bits required per channel use in order to generate an input that achieves arbitrarily accurate approximation of the output statistics for any given input process, is introduced. A general formula for resolvability that holds regardless of the channel memory structure, is obtained. It is shown that, for most channels, resolvability is equal to Shannon capacity. By-products of the analysis are a general formula for the minimum achievable (fixed-length) source coding rate of any finite-alphabet source, and a strong converse of the identification coding theorem, which holds for any channel that satisfies the strong converse of the channel coding theorem.

Polar codes for covert communications over asynchronous Discrete Memoryless Channels

“…Our code construction follows the idea put forward in [3,6], which suggests to have the code induce a "covert process" at the output of the adversary's channel by leveraging the notion of channel resolvability [8], and to show that the covert process is itself indistinguishable from the product distribution Q 0 .…”

On Evaluating the Rate-Distortion Function of Sources with Feed-Forward and the Capacity of Channels with Feedback

“…When we do not make any assumption on the nature of the joint process {X,X}, we need to use the information spectrum [10] version of (2). In particular, we will need the quantity…”

Universal Simulation with a Fidelity Criterion

“…In that sense, ρ n can be thought of as a generalized divergence. 1 Finally, let H(Y n |X m ) denote the conditional entropy of Y n given X m that is induced by the source P and the channel W (or, equivalently, the mapping ψ).…”

“…Simulation of sources and channels is a problem that has been studied in a series of works, see, e.g., [1], [7], [8], [9] and references therein. In all these works, it was assumed that the probability law of the desired process is perfectly known.…”