A simple technique for bounding the redundancy of source coding with side information

Abstract

A simple technique for bounding the redundancy of Slepian-Wolf coding is given. We demonstrate that our simple technique gives the tight bound established by He et al. Our proof is so simple that it can be easily extended to the case where the source (X n , Y n ) has an n-fold product distribution (i.e., (X1, Y1), . . . , (Xn, Yn) are independent but not necessarily identically distributed). It can be also applied to Wyner-AhlswedeKörner coding and gives novel bounds of the redundancies of the coding rates of the encoder and the helper.

Non-asymptotic and asymptotic analyses on Markov chains in several problems

“…So far, three kinds of asymptotic regimes have been studied in the information theory [1], [2], [17], [18], [19], [20], [21]:…”

Nonasymptotic and Second-Order Achievability Bounds for Coding With Side-Information

“…In the following, we will provide an inner bound to R WAK (n, ε) that improves on inner bounds that can be derived from previously obtained non-asymptotic bounds on P e (Φ n ) [6], [30].…”

“…Weak converses were proved in [2], [3] and a strong converse was proved in [29] using the "blowing-up lemma". An information spectrum characterization was provided by Miyake and Kanaya [8] and Kuzuoka [30] leveraged on the nonasymptotic bound which can be extracted from [8] to derive the redundancy for the WAK problem. Verdú [6] strengthened the non-asymptotic bound and showed that the error probability for the WAK problem is essentially bounded as…”

“…The proof of Theorem 12 follows directly from the finite blocklength bound in Corollary 6. In fact, the weaker bounds in [30] and [6] suffice for this purpose. Proof: Consider (57) and let us fix a process…”

Uniform Random Number Generation From Markov Chains: Non-Asymptotic and Asymptotic Analyses

“…B2-1) The large deviation regime in which the error probability ε asymptotically behaves like e −nr for some r > 0 [14], B2-2) The moderate deviation regime in which ε asymptotically behaves like e −n 1−2t r for some r > 0 and t ∈ (0, 1/2) [15], [16], [17], and B2-3) The second order regime in which ε is a constant [18], [4], [5], [6], [15], [16], [19]. We shall claim that a good non-asymptotic bound should be asymptotically optimal in at least one of the above mentioned three regimes.…”