Exact Channel Synthesis

Yu, Lei; Tan, Vincent Y. F.

doi:10.1109/isit.2019.8849592

Cited by 4 publications

(2 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, an alternative proof of the exact simulation result was given in [112] using a strengthened version of functional representation lemma [70], which is also applicable to infinite alphabet. Furthermore, the trade-off between the communication rate and the shared randomness rate for exact simulation was studied in [190].…”

Section: A the Reverse Shannon Theoremmentioning

confidence: 99%

Communication for Generating Correlation: A Unifying Survey

Sudan

Tyagi

Watanabe

2020

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

The task of manipulating correlated random variables in a distributed setting has received attention in the fields of both Information Theory and Computer Science. Often shared correlations can be converted, using a little amount of communication, into perfectly shared uniform random variables. Such perfect shared randomness, in turn, enables the solutions of many tasks. Even the reverse conversion of perfectly shared uniform randomness into variables with a desired form of correlation turns out to be insightful and technically useful. In this article, we describe progress-to-date on such problems and lay out pertinent measures, achievability results, limits of performance, and point to new directions.M. Sudan is with the 13 When X1, Z, and X2 form Markov chain in this order, the SK capacity is 0. 14 The benefit of feedback in the context of the wiretap channel was pointed out in [119].

show abstract

Section: A the Reverse Shannon Theoremmentioning

confidence: 99%

Communication for Generating Correlation: A Unifying Survey

Sudan

Tyagi

Watanabe

2020

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…If the closeness here is measured by the total variation (TV) distance between π n X P Y n |X n and the target joint distribution π n XY , this channel synthesis problem was investigated in [5]-[8] and the minimum communication rate was completely characterized by Cuff [7]. The exact synthesis of such a channel was considered in [9]- [12] where exact synthesis here means that the synthesized channel P Y n |X n is exactly equal to the target channel π n Y |X . The characterization of the minimum communication rate for exact synthesis (given the shared randomness rate) is an interesting but hard problem.…”

mentioning

confidence: 99%

Sequential Channel Synthesis

Lei¹,

Anantharam²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The channel synthesis problem has been widely investigated over the last decade. In this paper, we consider the sequential version in which the encoder and the decoder work in a sequential way. Under a mild assumption on the target joint distribution we provide a complete (single-letter) characterization of the solution for the point-to-point case, which shows that the canonical symbol-by-symbol mapping is not optimal in general, but is indeed optimal if we make some additional assumptions on the encoder and decoder. We also extend this result to the broadcast scenario and the interactive communication scenario. We provide bounds in the broadcast setting and a complete characterization of the solution under a mild condition on the target joint distribution in the interactive communication case. Our proofs are based on a Rényi entropy method. I. INTRODUCTIONThe study of the synthesis of distributions can be traced back to the seminal work by Wyner [1] where the problem studied was to characterize the smallest rate, in bits per symbol, at which common randomness needs to be provided to two agents, Alice and Bob, each having an arbitrary amount of private randomness, such that each of them can separately generate a sequence of random variables from respective finite sets, with the joint distribution being close to that of an i.i.d. sequence with a desired joint distribution at each symbol time (the notion of approximation in [1] is based on relative entropy). Wyner used this framework to define a notion of the common information of two dependent sources (the ones being synthesized by Alice and Bob respectively), which is known nowadays as Wyner's common information. This formulation is of considerable interest for problem of distributed control and game theory with distributed agents [2] because of the need to randomize for strategic reasons. It was generalized to the context of networks by Cuff et al. [3] where, in particular, the formulation allows for communication between the agents attempting to create i.i.d. copies of a target joint distribution, with the communication occurring at the level of blocks of symbols, see also [4]. For instance, for two agents, one can seek to find the minimum communication rate required for a pair of sender and receiver to synthesize a channel with a given input distribution in a distributed way. Specifically, the sender and receiver share a sequence of common random variables W n . After observing a source X n ∼ π n X and the common randomness W n , the sender generates bits and send them to the receiver, who generates another source Y n according to the common randomness W n and the bits that he/she receives. They cooperate in such a way so that the channel induced by the code P Y n |X n is close to a target channel π n Y |X . If the closeness here is measured by the total variation (TV) distance between π n X P Y n |X n and the target joint distribution π n XY , this channel synthesis problem was investigated in [5]-[8] and the minimum communication rate was completely charact...

show abstract

On Exact and ∞-Rényi Common Informations

Tan

2020

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Recently, two extensions of Wyner's common information -exact and Rényi common informations -were introduced respectively by Kumar, Li, and El Gamal (KLE), and the present authors. The class of common information problems refers to determining the minimum rate of the common input to two independent processors needed to generate an exact or approximate joint distribution. For the exact common information problem, an exact generation of the target distribution is required, while for Wyner's and α-Rényi common informations, the relative entropy and Rényi divergence with order α were respectively used to quantify the discrepancy between the synthesized and target distributions. The exact common information is larger than or equal to Wyner's common information. However, it was hitherto unknown whether the former is strictly larger than the latter. In this paper, we first establish the equivalence between the exact and ∞-Rényi common informations, and then provide single-letter upper and lower bounds for these two quantities. For doubly symmetric binary sources, we show that the upper and lower bounds coincide, which implies that for such sources, the exact and ∞-Rényi common informations are completely characterized. Interestingly, we observe that for such sources, these two common informations are strictly larger than Wyner's. This answers an open problem posed by KLE. Furthermore, we extend Wyner's, ∞-Rényi, and exact common informations to sources with countably infinite or continuous alphabets, including Gaussian sources. Index TermsWyner's common information, Rényi common information, Exact common information, Exact channel simulation, Exact source simulation, Communication complexity of correlation Recently, the present authors [2], [3] introduced the notion of Rényi common information, which is defined as the minimum common rate when the KL divergence is replaced by more general divergences -the family of Rényi divergences. When s = 0, Rényi common information reduces to Wyner's common information. We proved that for Rényi divergences of order 1 + s ∈ (0, 1], the minimum rate needed to guarantee that the (normalized and unnormalized) Rényi divergences vanish asymptotically is equal to Wyner's common information. However, for Rényi divergences of order 1 + s ∈ (1, 2], we only provided an upper bound, which is larger than Wyner's common information in general. Furthermore, the common information with approximation error measured by the total variation (TV) distance is also equal to Wyner's common information [2], [4], [5]; and exponential achievability and converse results was established in [2], [4], [6].Kumar, Li, and El Gamal (KLE) [5] extended Wyner's common information in a different way. They assumed variable-length codes and exact generation of the correlated sources (X, Y ) ∼ π XY , instead of block codes and approximate simulation of π XY as assumed by Wyner [1] and by us [2], [3]. For such exact generation problem, KLE [5] characterized the minimum common rate, coined exact common information, by T Exac...

show abstract

Exact Channel Synthesis

Cited by 4 publications

References 34 publications

Communication for Generating Correlation: A Unifying Survey

Communication for Generating Correlation: A Unifying Survey

Sequential Channel Synthesis

On Exact and ∞-Rényi Common Informations

Contact Info

Product

Resources

About