A Generalization of the Rate-Distortion Function for Wyner-Ziv Coding of Noisy Sources in the Quadratic-Gaussian Case

Abstract

We extend the rate-distortion function for Wyner-Ziv coding of noisy sources with quadratic distortion, in the jointly Gaussian case, to more general statistics. It suffices that the noisy observation Z be the sum of a function of the side information Y and independent Gaussian noise, while the source data X must be the sum of a function of Y , a linear function of Z, and a random variable N such that the conditional expectation of N given Y and Z is zero, almost surely. Furthermore, the side information Y may be arbitrarily distributed in any alphabet, discrete or continuous. Under these general conditions, we prove that no rate loss is incurred due to the unavailability of the side information at the encoder. In the noiseless Wyner-Ziv case, i. e., when the source data is directly observed, the assumptions are still less restrictive than those recently established in the literature. We confirm, theoretically and experimentally, the consistency of this analysis with some of the main results on high-rate Wyner-Ziv quantization of noisy sources.

High-rate quantization and transform coding with side information at the decoder

“…Taking expectation of the conditional costs proves that (8) and (9) are valid for the conditional quantizer of Fig. 7.…”

“…Since this modified structure is more constrained than the general structure, its performance may be degraded. However, the design of the noisy WZ scalar quantizers at each subband, for instance using the extension of the Lloyd algorithm in [8], may be simpler than the implementation of a nonlinear vector estimatorx Z (z), or a noisy WZ vector quantizer operating directly on the noisy observation vector.…”

“…7. The validity of (10) for the conditional quantizer can be shown by solving for V in (9) and substituting the result into (8).…”

“…Under much more restrictive statistical conditions, this also holds for coding of noisy observations of unseen data [7,8].…”

“…The nondistributed case was studied in [40][41][42], and [7,[43][44][45]8] analyzed the distributed case from an information-theoretic point of view. Using Gaussian statistics and Mean-Squared Error (MSE) as a distortion measure, [13] proved that distributed coding of two noisy observations without side information can be carried out with a performance close to that of joint coding and denoising, in the limit of small distortion and large dimension.…”