A novel definition of the conditional smooth Rényi entropy, which is different from that of Renner and Wolf, is introduced. It is shown that our definition of the conditional smooth Rényi entropy is appropriate to give lower and upper bounds on the optimal guessing moment in a guessing problem where the guesser is allowed to stop guessing and declare an error. Further a general formula for the optimal guessing exponent is given. In particular, a single-letterized formula for mixture of i.i.d. sources is obtained. Another application in the problem of source coding with the common side-information available at the encoder and decoder is also demonstrated. Index Termsguessing, information-spectrum method, side information, source coding, the conditional smooth Rényi entropy

Abstract-We present novel non-asymptotic or finite blocklength achievability bounds for three side-information problems in network information theory. These include (i) the WynerAhlswede-Körner (WAK) problem of almost-lossless source coding with rate-limited side-information, (ii) the Wyner-Ziv (WZ) problem of lossy source coding with side-information at the decoder and (iii) the Gel'fand-Pinsker (GP) problem of channel coding with noncausal state information available at the encoder. The bounds are proved using ideas from channel simulation and channel resolvability. Our bounds for all three problems improve on all previous non-asymptotic bounds on the error probability of the WAK, WZ and GP problems-in particular those derived by Verdú. Using our novel non-asymptotic bounds, we recover the general formulas for the optimal rates of these side-information problems. Finally, we also present achievable second-order coding rates by applying the multidimensional Berry-Esséen theorem to our new non-asymptotic bounds. Numerical results show that the second-order coding rates obtained using our non-asymptotic achievability bounds are superior to those obtained using existing finite blocklength bounds.

We present novel non-asymptotic or finite blocklength achievability bounds for three side-information problems in network information theory. These include (i) the Wyner-Ahlswede-Körner (WAK) problem of almost-lossless source coding with rate-limited side-information, (ii) the Wyner-Ziv (WZ) problem of lossy source coding with side-information at the decoder and (iii) the Gel'fand-Pinsker (GP) problem of channel coding with noncausal state information available at the encoder. The bounds are proved using ideas from channel simulation and channel resolvability. Our bounds for all three problems improve on all previous non-asymptotic bounds on the error probability of the WAK, WZ and GP problems-in particular those derived by Verdú. Using our novel non-asymptotic bounds, we recover the general formulas for the optimal rates of these side-information problems. Finally, we also present achievable second-order coding rates by applying the multidimensional Berry-Esséen theorem to our new non-asymptotic bounds. Numerical results show that the second-order coding rates obtained using our non-asymptotic achievability bounds are superior to those obtained using existing finite blocklength bounds.

The problem of distributed data compression for function computation is considered, where (i) the function to be computed is not necessarily symbol-wise function and (ii) the information source has memory and may not be stationary nor ergodic. We introduce the class of smooth sources and give a sufficient condition on functions so that the achievable rate region for computing coincides with the Slepian-Wolf region (i.e., the rate region for reproducing the entire source) for any smooth sources. Moreover, for symbol-wise functions, the necessary and sufficient condition for the coincidence is established. Our result for the full side-information case is a generalization of the result by Ahlswede and Csiszár to sources with memory; our dichotomy theorem is different from Han and Kobayashi's dichotomy theorem, which reveals an effect of memory in distributed function computation. All results are given not only for fixed-length coding but also for variable-length coding in a unified manner. Furthermore, for the full side-information case, the error probability in the moderate deviation regime is also investigated. Index Termsdistributed computing, information-spectrum method, Slepian-Wolf coding I. INTRODUCTIONWe study the problem of distributed data compression for function computation described in Fig. 1 and Fig. 2, where the function to be computed is not necessarily symbol-wise function. In [1], Körner and Marton revealed that the achievable rate region for computing modulo-sum is strictly larger than the rate region that can be achieved by first applying Slepian-Wolf coding [2] and then computing the function. 1 Since then, distributed coding schemes that are tailored for some classes of functions were studied (e.g., see [3, Chapter 21]). These results are the cases such that the structure of functions can be utilized for distributed coding. However, not all functions have such nice structures, and even for some classes of functions, it is known that the Slepian-Wolf region cannot be improved at

A simple technique for bounding the redundancy of Slepian-Wolf coding is given. We demonstrate that our simple technique gives the tight bound established by He et al. Our proof is so simple that it can be easily extended to the case where the source (X n , Y n ) has an n-fold product distribution (i.e., (X1, Y1), . . . , (Xn, Yn) are independent but not necessarily identically distributed). It can be also applied to Wyner-AhlswedeKörner coding and gives novel bounds of the redundancies of the coding rates of the encoder and the helper.