Distributed Simulation of Continuous Random Variables

Abstract

We establish the first known upper bound on the exact and Wyner's common information of n continuous random variables in terms of the dual total correlation between them (which is a generalization of mutual information). In particular, we show that when the pdf of the random variables is log-concave, there is a constant gap of n 2 log e + 9n log n between this upper bound and the dual total correlation lower bound that does not depend on the distribution. The upper bound is obtained using a computationally efficient dyadic decomposition scheme for constructing a discrete common randomness variable W from which the n random variables can be simulated in a distributed manner. We then bound the entropy of W using a new measure, which we refer to as the erosion entropy.

Communication for Generating Correlation: A Unifying Survey

“…Alternatively, we can consider the expected length of common randomness required to generate the target joint distribution exactly. Such a variant of the problem, termed exact common information, was studied in [109] (see also [111] for a protocol that exactly generates target distributions on continuous alphabets). The exact common information is larger than or equal to the Wyner common information by definition.…”

Exact Channel Synthesis

“…However, in this paper (as well as in [7]), we combine it with distribution truncation techniques to analyze sources with countably infinite alphabets. We also combine the mixture decomposition technique with truncation, discretization, and Li and El Gamal's dyadic decomposition technique [21] to analyze continuous sources. Furthermore, as by-products of our analyses, various lemmas that may be of independent interest are derived, e.g., the "chain rule for coupling" lemma, the (distributed and centralized) Rényi-covering lemmas, the log-concavity invariance lemma, etc.…”

“…where equality in (21) holds if and only if π XY = π X π Y . b) Moreover, assume supp(π XY ) = X ×Y.…”

“…In [7, Corollaries 2 and 3], we applied techniques involving truncation, mixture decomposition, discretization, and dyadic decomposition (studied in [21]) to show that 8 inf…”

“…then there must exist a sequence of variable-length codes with rates (R 0 , R) that exactly generates π n XY . Proof of Lemma 1: The proof follows similar ideas as the one of [7,Lemma 2], where the techniques of mixture decomposition, truncation, discretization, and Li and El Gamal's dyadic decomposition [21] were used. However, the difference is that in [7, Lemma 2], we used the mixture decomposition technique to decompose a memoryless correlated source π n XY , but here we need to decompose a memoryless channel π n Y |X .…”