Extended Gray-Wyner system with complementary causal side information

Abstract

We establish the rate region of an extended Gray-Wyner system for 2-DMS (X, Y ) with two additional decoders having complementary causal side information. This extension is interesting because in addition to the operationally significant extreme points of the Gray-Wyner rate region, which include Wyner's common information, Gács-Körner common information and information bottleneck, the rate region for the extended system also includes the Körner graph entropy, the privacy funnel and excess functional information, as well as three new quantities of potential interest, as extreme points. To simplify the investigation of the 5-dimensional rate region of the extended Gray-Wyner system, we establish an equivalence of this region to a 3-dimensional mutual information region that consists of the set of all triples of the form (I(X; U ), I(Y ; U ), I(X, Y ; U )) for some p U |X,Y . We further show that projections of this mutual information region yield the rate regions for many settings involving a 2-DMS, including lossless source coding with causal side information, distributed channel synthesis, and lossless source coding with a helper. Index TermsGray-Wyner system, side information, complementary delivery, Körner graph entropy, privacy funnel. I. INTRODUCTIONThe lossless Gray-Wyner system [1] is a multi-terminal source coding setting for two discrete memoryless source (2-DMS) (X, Y ) with one encoder and two decoders. This setup draws some of its significance from providing operational interpretation for several information theoretic quantities of interest, namely Wyner's common information [2], the Gács-Körner common information [3], the necessary conditional entropy [4], and the information bottleneck [5].In this paper, we consider an extension of the Gray-Wyner system (henceforth called the EGW system), which includes two new individual descriptions and two decoders with causal side information as depicted in Figure 1. The encoder maps sequences from a 2-DMS (X, Y ) into five indices M i ∈ [1 : 2 nRi ], i = 0, . . . , 4. Decoders 1 and 2 correspond to those of the Gray-Wyner system, that is, decoder 1 recovers X n from (M 0 , M 1 ) and decoder 2 recovers Y n from (M 0 , M 2 ). At time i ∈ [1 : n], decoder 3 recovers X i causally from (M 0 , M 3 , Y i ) and decoder 4 similarly recovers Y i causally from (M 0 , M 4 , X i ). Note that decoders 3 and 4 correspond to those of the complementary delivery setup studied in [6], [7] with causal (instead of noncausal) side information and with two additional private indices M 3 and M 4 . This extended Gray-Wyner system setup is lossless, that is, the decoders recover their respective source sequences with probability of error that vanishes as n approaches infinity. The rate region R of the EGW system is defined in the usual way as the closure of the set of achievable rate tuplesThe first contribution of this paper is to establish the rate region of the EGW system. Moreover, to simplify the study of this rate region and its extreme points, we show that it is equivale...

Bottleneck Problems: An Information and Estimation-Theoretic View

“…This in fact delineate an important difference between and : while describes the entire rate-region of an information-theoretic setup, specifies only a corner point of a rate region. Other information-theoretic settings related to and include CEO problem [ 91 ] and source coding for the Gray-Wyner network [ 92 ].…”

Common Information, Noise Stability, and Their Extensions

“…Further generalizations of GKW's common information to more complicated networks have also been investigated in the literature; see for example the comprehensive surveys by Sudan, Tyagi, and Watanabe [158], Liang, Poor, and Shamai [111], and Bloch and Barros [21]. Other interesting quantities defined based on Gray-Wyner system, such as the Körner graph entropy [99], the privacy funnel [116], and the excess functional information [109], can be found in Li and El Gamal [108]. These quantities can also considered as generalizations of GKW's and Wyner's common information.…”

Privacy-aware guessing efficiency

“…3. The optimal privacy filter for h 2 (ε) for ε ∈ [ε L ,ᾱ), where ζ2(ε) is defined in (6). h(P X n Y n , ε), as implied by the following theorem.…”

“…In [4], an equivalent result was obtained in terms of the singular values of the operator f → E[f (X)|Y ]. Although a connection between this information-theoretic privacy measure and a coding theorem is established in [2] and [6], the use of mutual information as a privacy measure is not satisfactorily motivated in an operational sense. To find a measure of privacy with a clear operational meaning, in this paper we take an estimation-theoretic approach and define both privacy and utility measures in terms of the probability of guessing correctly.…”