Sequential Channel Synthesis

Abstract

The channel synthesis problem has been widely investigated over the last decade. In this paper, we consider the sequential version in which the encoder and the decoder work in a sequential way. Under a mild assumption on the target joint distribution we provide a complete (single-letter) characterization of the solution for the point-to-point case, which shows that the canonical symbol-by-symbol mapping is not optimal in general, but is indeed optimal if we make some additional assumptions on the encoder and decoder. We also extend this result to the broadcast scenario and the interactive communication scenario. We provide bounds in the broadcast setting and a complete characterization of the solution under a mild condition on the target joint distribution in the interactive communication case. Our proofs are based on a Rényi entropy method. I. INTRODUCTIONThe study of the synthesis of distributions can be traced back to the seminal work by Wyner [1] where the problem studied was to characterize the smallest rate, in bits per symbol, at which common randomness needs to be provided to two agents, Alice and Bob, each having an arbitrary amount of private randomness, such that each of them can separately generate a sequence of random variables from respective finite sets, with the joint distribution being close to that of an i.i.d. sequence with a desired joint distribution at each symbol time (the notion of approximation in [1] is based on relative entropy). Wyner used this framework to define a notion of the common information of two dependent sources (the ones being synthesized by Alice and Bob respectively), which is known nowadays as Wyner's common information. This formulation is of considerable interest for problem of distributed control and game theory with distributed agents [2] because of the need to randomize for strategic reasons. It was generalized to the context of networks by Cuff et al. [3] where, in particular, the formulation allows for communication between the agents attempting to create i.i.d. copies of a target joint distribution, with the communication occurring at the level of blocks of symbols, see also [4]. For instance, for two agents, one can seek to find the minimum communication rate required for a pair of sender and receiver to synthesize a channel with a given input distribution in a distributed way. Specifically, the sender and receiver share a sequence of common random variables W n . After observing a source X n ∼ π n X and the common randomness W n , the sender generates bits and send them to the receiver, who generates another source Y n according to the common randomness W n and the bits that he/she receives. They cooperate in such a way so that the channel induced by the code P Y n |X n is close to a target channel π n Y |X . If the closeness here is measured by the total variation (TV) distance between π n X P Y n |X n and the target joint distribution π n XY , this channel synthesis problem was investigated in [5]-[8] and the minimum communication rate was completely charact...