A Lossy Source Coding Interpretation of Wyner’s Common Information

Xu, Guangning; Liu, Wei; Chen, Biao

doi:10.1109/tit.2015.2506560

Cited by 37 publications

(72 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The rest of this section and the remaining of the paper is organized as follows. In Section I-A we introduced the three concepts which are further described in Charalambous and van Schuppen [3], in Sections I-B-I-C we recall the Gray and Wyner characterization of the rate region [4], and the characterization of the minimum lossy common message rate on the Gray and Wyner rate region due to Viswanatha, Akyol and Rose [5], and Xu, Liu, and Chen [6]. In Section II we present our main results in the form of theorems.…”

Section: Introduction Main Concepts Literature Main Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Characterization of Conditional Independence and Weak Realizations of Multivariate Gaussian Random Variables: Applications to Networks

Charalambous¹,

Schuppen²

2020

Preprint

View full text Add to dashboard Cite

The Gray and Wyner lossy source coding for a simple network for sources that generate a tuple of jointly Gaussian random variables (RVs) X 1 : Ω → R p 1 and X 2 : Ω → R p 2 , with respect to square-error distortion at the two decoders is reexamined using (1) Hotelling's geometric approach of Gaussian RVs-the canonical variable form, and (2) van Putten's and van Schuppen's parametrization of joint distributions P X 1 ,X 2 ,W by Gaussian RVs W : Ω → R n which make (X 1 , X 2 ) conditionally independent, and the weak stochastic realization of (X 1 , X 2 ). Item ( 2) is used to parametrize the lossy rate region of the Gray and Wyner source coding problem for joint decoding with mean-square error distortions, by the covariance matrix of RV W . From this then follows Wyner's common information C W (X 1 , X 2 ) (information definition) is achieved by W with identity covariance matrix, while a formula for Wyner's lossy common information (operational definition) is derived, given by

show abstract

Section: Introduction Main Concepts Literature Main Resultsmentioning

confidence: 99%

“…Viswanatha, Akyol, and Rose [5], and Xu, Liu, and Chen [6], characterized the minimum lossy common message rate on the rate region R GW (∆ 1 , ∆ 2 ), as follows.…”

Section: Wyner's Lossy Common Informationmentioning

confidence: 99%

Characterization of Conditional Independence and Weak Realizations of Multivariate Gaussian Random Variables: Applications to Networks

Charalambous¹,

Schuppen²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…We will argue that caching without a bias in user preference is closely related to the notions of Wyner's common information [9] and Watanabe's total correlation [10]. Research that is closely related to this work therefore also includes the work of Viswanatha et al [11] and Xu et al [12]. These authors have introduced and studied notions of lossy common information, as well as provided some important characterizations of these properties for Gaussian distributions.…”

Section: The Most Popular Caching Paper By Maddah-ali and Niesenmentioning

confidence: 86%

“…The Gray-Wyner network was introduced originally in [8], but also featured more recently in an explicit lossy source coding setting [11], [12]. The region of achievable ratedistortion tuples on the Gray-Wyner network is the union of all…”

Section: B Analogy To the Gray-wyner Networkmentioning

confidence: 99%

Caching (Bivariate) Gaussians

Veld

Gastpar

2020

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Caching is a technique that alleviates networks during peak hours by transmitting partial information before a request for any is made. In a lossy setting of Gaussian databases, we study a single-user model in which good caching strategies minimize the data still needed on average once the user requests a file. The encoder decides on a caching strategy by weighing the benefit from two key parameters: the prior preference for a file and the correlation among the files. Considering uniform prior preference but correlated files, caching becomes an application of Wyner's common information and Watanabe's total correlation. We show this case triggers a split: caching Gaussian sources is a non-convex optimization problem unless one spends enough rate to cache all the common information between files. Combining both correlation and user preference we explicitly characterize the full trade-off when the encoder uses Gaussian codebooks in a database of two files: we show that as the size of the cache increases, the encoder should change strategy and increasingly prioritize user preference over correlation. In this specific case we also address the loss in performance incurred if the encoder has no knowledge of the user's preference and show that this loss is bounded.

show abstract

“…Note that, for this example, I(X; Y) = 1 2 log 1 1−ρ 2 . This case was solved in [25,26] using a parameterization of conditionally independent distributions, and we have recently found an alternative proof that also extends to the generalization of Wyner's common information discussed in the next subsection [5].…”

Section: Definition 1 ([23]mentioning

confidence: 99%

Common Information Components Analysis

Sula

Gastpar

2021

Entropy

View full text Add to dashboard Cite

Wyner’s common information is a measure that quantifies and assesses the commonality between two random variables. Based on this, we introduce a novel two-step procedure to construct features from data, referred to as Common Information Components Analysis (CICA). The first step can be interpreted as an extraction of Wyner’s common information. The second step is a form of back-projection of the common information onto the original variables, leading to the extracted features. A free parameter γ controls the complexity of the extracted features. We establish that, in the case of Gaussian statistics, CICA precisely reduces to Canonical Correlation Analysis (CCA), where the parameter γ determines the number of CCA components that are extracted. In this sense, we establish a novel rigorous connection between information measures and CCA, and CICA is a strict generalization of the latter. It is shown that CICA has several desirable features, including a natural extension to beyond just two data sets.

show abstract

A Lossy Source Coding Interpretation of Wyner’s Common Information

Cited by 37 publications

References 27 publications

Characterization of Conditional Independence and Weak Realizations of Multivariate Gaussian Random Variables: Applications to Networks

Characterization of Conditional Independence and Weak Realizations of Multivariate Gaussian Random Variables: Applications to Networks

Caching (Bivariate) Gaussians

Common Information Components Analysis

Contact Info

Product

Resources

About