Every bad code has a good subcode: A local converse to the coding theorem

Ahlswede, Rudolf; Dueck, Gunter

doi:10.1007/bf00535683

Cited by 19 publications

(13 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the fact that E P Y n (V ) = P (C) Y n , convexity of the relative entropy, and (33), we get 1 Note that the gain in using instead the inequality n nδn ≤ exp n h(δn) is marginal, and it does not have any advantage asymptotically for large n.…”

Section: From This and (18) It Follows Thatmentioning

confidence: 96%

“…In fact, the convergence in (1) holds not just for DMC's, but for arbitrary channels satisfying the condition C = lim n→∞ 1 n sup P X n ∈P(X n ) I(X n ; Y n ). In a recent preprint [10], Polyanskiy and Verdú extended the results of [14] for codes with nonvanishing probability of error, provided the maximal probability of error criterion and deterministic encoders are used.…”

Section: Introductionmentioning

confidence: 93%

See 1 more Smart Citation

Refined bounds on the empirical distribution of good channel codes via concentration inequalities

Raginsky

Sason

2013

2013 IEEE International Symposium on Information Theory

View full text Add to dashboard Cite

We derive sharpened inequalities on the empirical output distribution of good channel codes with deterministic encoders and with non-vanishing maximal probability of decoding error. These inequalities refine recent bounds of Polyanskiy and Verdú by identifying closed-form expressions for certain asymptotic terms, which facilitates their calculation for finite blocklengths. The analysis relies on concentration-of-measure inequalities, specifically on McDiarmid's method of bounded differences and its close ties to transportation inequalities for weighted Hamming metrics. An operational implication of the new bounds is addressed.

show abstract

Section: From This and (18) It Follows Thatmentioning

confidence: 96%

Section: Introductionmentioning

confidence: 93%

Refined bounds on the empirical distribution of good channel codes via concentration inequalities

Raginsky

Sason

2013

2013 IEEE International Symposium on Information Theory

View full text Add to dashboard Cite

show abstract

“…In terms of applications Ahlswede [15] used the blowing up lemma to prove a local strong converse for maximal error codes over a two-terminal DMC, showing that all bad codes have a good subcode of almost the same rate. Using the same lemma, Körner and Marton [16] developed a general framework for determining the achievable rates of a number of source and channel networks.…”

Section: A Images and Quasi-imagesmentioning

confidence: 99%

“…for all u ∈Ũ, where ε n = n −1 |X ||Y |+1 by Corollary 3 15. The set D (saturate),∅ (U, p Y |X ; ν n ) is not considered because the random variable ∅ is trivially uniform by convention.…”

Section: A One Way Communications Over a Dmcmentioning

confidence: 99%

Inducing Information Stability to Obtain Information Theoretic Necessary Requirements

Graves

Wong

2020

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

This work constructs a discrete random variable that, when conditioned upon, ensures information stability of quasi-images. Using this construction, a new methodology is derived to obtain information theoretic necessary conditions directly from operational requirements. In particular, this methodology is used to derive new necessary conditions for keyed authentication over discrete memoryless channels and to establish the capacity region of the wiretap channel, subject to finite leakage and finite error, under two different secrecy metrics. These examples establish the usefulness of the proposed methodology.The set of all valid empirical distributions for an n-length sequence will be denoted P n . For empirical conditional distributions we shall use P n (Y|p) where p ∈ P n (X ), to denote the set of conditional empirical distributions w for which w(y|x)p(x) is a valid distribution in P(Y, X ).Many of the results to be presented in later sections involves DRVs that satisfy specific sets of relationship and/or properties.For relationships between DRVs in particular, we will use the following two operators. First if X ❝ Y ❝ Z, then DRVs X, Y, Z form a Markov chain in that order. In other words p X,Y,Z (x, y, z) = p X (x)p Y |X (y|x)p Z|Y (z|y) for all (x, y, z) ∈ X × Y × Z. On the other hand, if X ≫ Y , then Y can be written as a deterministic function of X. For any DRVs X, Y, Z, ifTo simplify the statements of our results, we will adopt the standard set notation when describing DRVs satisfying a specific set of properties. For instance, the DRVs U, X, Y that satisfy the conditions that |U| ≤ n and that U ❝ X ❝ Y will be denoted by (U, X, Y ) : {|U| ≤ n, U ❝ X ❝ Y }.

show abstract

“…Informally, this corollary of the blowing-up lemma says that "any bad code contains a good subcode." Using this result, Ahlswede and Dueck [46] established a strong converse for channel coding as follows: Consider an (n, M, ε)-code C = {(u j , D j )} M j=1 . Each decoding set D j can be "blown up" to a setD j ⊆ Y n with…”

Section: Lemmamentioning

confidence: 99%

Concentration of Measure Inequalities in Information Theory, Communications, and Coding

Raginsky

Sason

2006

View full text Add to dashboard Cite

During the last two decades, concentration of measure has been a subject of various exciting developments in convex geometry, functional analysis, statistical physics, high-dimensional statistics, probability theory, information theory, communications and coding theory, computer science, and learning theory. One common theme which emerges in these fields is probabilistic stability: complicated, nonlinear functions of a large number of independent or weakly dependent random variables often tend to concentrate sharply around their expected values. Information theory plays a key role in the derivation of concentration inequalities. Indeed, both the entropy method and the approach based on transportation-cost inequalities are two major information-theoretic paths toward proving concentration.This brief survey is based on a recent monograph of the authors in the Foundations and Trends in Communications and Information Theory, and a tutorial given by the authors at ISIT 2015. It introduces information theorists to three main techniques for deriving concentration inequalities: the martingale method, the entropy method, and the transportation-cost inequalities. Some applications in information theory, communications, and coding theory are used to illustrate the main ideas.

show abstract

Every bad code has a good subcode: A local converse to the coding theorem

Cited by 19 publications

References 6 publications

Refined bounds on the empirical distribution of good channel codes via concentration inequalities

Refined bounds on the empirical distribution of good channel codes via concentration inequalities

Inducing Information Stability to Obtain Information Theoretic Necessary Requirements

Concentration of Measure Inequalities in Information Theory, Communications, and Coding

Contact Info

Product

Resources

About