A technique for deriving one-shot achievability results in network information theory

Abstract

This paper proposes a novel technique to prove a one-shot version of achievability results in network information theory. The technique is not based on covering and packing lemmas. In this technique, we use an stochastic encoder and decoder with a particular structure for coding that resembles both the ML and the joint-typicality coders. Although stochastic encoders and decoders do not usually enhance the capacity region, their use simplifies the analysis. The Jensen inequality lies at the heart of error analysis, which enables us to deal with the expectation of many terms coming from stochastic encoders and decoders at once. The technique is illustrated via several examples: pointto-point channel coding, Gelfand-Pinsker, Broadcast channel (Marton), Berger-Tung, Heegard-Berger/Kaspi, Multiple description coding and Joint source-channel coding over a MAC. Most of our one-shot results are new. The asymptotic forms of these expressions is the same as that of classical results. Our one-shot bounds in conjunction with multidimensional Berry-Essen CLT imply new results in the finite blocklength regime. In particular applying the one-shot result for the memoryless broadcast channel in the asymptotic case, we get the entire region of Marton's inner bound without any need for time-sharing.March 5, 2013 DRAFT 3 Definition 2. For a pmf p X,Y,Z , the conditional information density ı(x; y|z) is defined byand for general r.v.'s it is defined byWhenever the underlying distribution is clear from the context, we drop the subscript p from ı p (x; y|z).Definition 3. Let X be a multi-dimensional normal variable with zero mean and covariance matrix V. The complementary multivariate Gaussian cumulative distribution region associated with V is defined byWe use M and J to denote size of alphabets of random variables M and J, respectively, i.e. M = |M| and J = |J |. All the logarithms are in base two throughout this paper. III. MULTI-TERMINAL CHANNEL CODING PROBLEMSTo illustrate the application of our technique to multi-terminal channel coding problems, we study the problems of point-to-point channel, Gelfand-Pinsker and Broadcast channels (Marton) in this section. A. Point-to-point channelWe begin our illustration of the one-shot achievability proof with the classical point-to-point channel. Consider a channel with the law q Y |X and an input distribution q X . Let C = {X(1), · · · , X(M)} be a random codebook where the elements X(i) are drawn independently from q X (each codeword X(i) is only a single rv). As usual, X(m) is the codeword used for transmission of the message m. For the decoding we use an stochastic variation of MAP decoding.Instead of declaring the messagem with maximal posterior probability as in MAP, the decoder randomly draws a messagem from the conditional pmf P M|Y , where P is the induced pmf by the code, P M,Y (m, y) = 1 M q(y|X(m)). 1 More specifically, P M|Y (m|y) = q(y|X(m)) m q(y|X(m)) = 2 ıq(y;X(m)) m 2 ıq(y;X(m)) .(1)The mutual information term ı q (y; X(m)) is computed using pmf q X q Y |X that has nothing...

The likelihood decoder: Error exponents and mismatch

“…The error probability is given by p e (C) = P[m = m], where the probability is over the message, the channel, and the decoder. Setting q(x, y) = W (y|x) recovers the decoder studied in [4].…”

“…ACHIEVABILITY VIA DUAL-DOMAIN ANALYSIS The following theorem presents a non-asymptotic bound obtained via a refinement of the analysis of Yassaee et al [4], with an aim to improve the resulting error exponent. The main difference is the introduction of a parameter s ∈ (0, 1] that can be optimized.…”

“…Another alternative that has recently gained interest is the likelihood decoder [4], which is a stochastic decoder such that the probability of choosing a given codeword is proportional to its likelihood under the channel law. This decoder has been shown to simplify the derivations of a variety of asymptotic achievability bounds in network information theory [4], and analogous likelihood encoders have proved to be useful in the context of lossy compression [5]. The likelihood decoder is a special case of the pretty good measurement in quantum information theory [6], [7].…”

“…For the point-to-point channel coding problem, it was shown in [4] that this decoder not only yields the channel capacity of discrete memoryless channels (DMCs), but also the channel dispersion [8], [9]. On the other hand, existing bounds are not powerful enough to attain the best known random-coding error exponents [2], [10] in general.…”

“…On the other hand, existing bounds are not powerful enough to attain the best known random-coding error exponents [2], [10] in general. One of our contributions is a refined analysis of that in [4] that yields the random-coding error exponents of optimal ML decoding for both i.i.d. and constant-composition random coding.…”