Extended Gray–Wyner System With Complementary Causal Side Information

Abstract

We establish the rate region of an extended Gray-Wyner system for 2-DMS (X, Y ) with two additional decoders having complementary causal side information. This extension is interesting because in addition to the operationally significant extreme points of the Gray-Wyner rate region, which include Wyner's common information, Gács-Körner common information and information bottleneck, the rate region for the extended system also includes the Körner graph entropy, the privacy funnel and excess functional information, as well as three new quantities of potential interest, as extreme points. To simplify the investigation of the 5-dimensional rate region of the extended Gray-Wyner system, we establish an equivalence of this region to a 3-dimensional mutual information region that consists of the set of all triples of the form (I(X; U ), I(Y ; U ), I(X, Y ; U )) for some p U |X,Y . We further show that projections of this mutual information region yield the rate regions for many settings involving a 2-DMS, including lossless source coding with causal side information, distributed channel synthesis, and lossless source coding with a helper. Index TermsGray-Wyner system, side information, complementary delivery, Körner graph entropy, privacy funnel. I. INTRODUCTIONThe lossless Gray-Wyner system [1] is a multi-terminal source coding setting for two discrete memoryless source (2-DMS) (X, Y ) with one encoder and two decoders. This setup draws some of its significance from providing operational interpretation for several information theoretic quantities of interest, namely Wyner's common information [2], the Gács-Körner common information [3], the necessary conditional entropy [4], and the information bottleneck [5].In this paper, we consider an extension of the Gray-Wyner system (henceforth called the EGW system), which includes two new individual descriptions and two decoders with causal side information as depicted in Figure 1. The encoder maps sequences from a 2-DMS (X, Y ) into five indices M i ∈ [1 : 2 nRi ], i = 0, . . . , 4. Decoders 1 and 2 correspond to those of the Gray-Wyner system, that is, decoder 1 recovers X n from (M 0 , M 1 ) and decoder 2 recovers Y n from (M 0 , M 2 ). At time i ∈ [1 : n], decoder 3 recovers X i causally from (M 0 , M 3 , Y i ) and decoder 4 similarly recovers Y i causally from (M 0 , M 4 , X i ). Note that decoders 3 and 4 correspond to those of the complementary delivery setup studied in [6], [7] with causal (instead of noncausal) side information and with two additional private indices M 3 and M 4 . This extended Gray-Wyner system setup is lossless, that is, the decoders recover their respective source sequences with probability of error that vanishes as n approaches infinity. The rate region R of the EGW system is defined in the usual way as the closure of the set of achievable rate tuplesThe first contribution of this paper is to establish the rate region of the EGW system. Moreover, to simplify the study of this rate region and its extreme points, we show that it is equivale...

Bottleneck Problems: An Information and Estimation-Theoretic View

“…This in fact delineate an important difference between IB and PF: while IB describes the entire rate-region of an information-theoretic setup, PF specifies only a corner point of a rate region. Other information-theoretic settings related to IB and PF include CEO problem [91] and source coding for the Gray-Wyner network [92].…”