Reliability in Source Coding With Side Information

Abstract

We study error exponents for source coding with side information. Both achievable exponents and converse bounds are obtained for the following two cases: lossless source coding with coded information (SCCSI) and lossy source coding with full side information (Wyner-Ziv). These results recover and extend several existing results on source-coding error exponents and are tight in some circumstances. Our bounds have a natural interpretation as a two-player game between nature and the code designer, with nature seeking to minimize the exponent and the code designer seeking to maximize it. In the Wyner-Ziv problem our analysis exposes a tension in the choice of test channel with the optimal test channel balancing two competing error events. The Gaussian and binary-erasure cases are examined in detail. I. INTRODUCTIONIn a typical lossy data compression problem a source is to be compressed by an encoder at a prescribed rate so that a decoder may reproduce the source to within some desired fidelity (distortion). Sometimes present, in addition to the data to be compressed, is some correlated information that can be utilized by a second encoder, that is able to send a separate message to the decoder. We refer to this kind of problem as source coding with side information (SCSI). The set-up is depicted in Fig. 1, where a source X is compressed by encoder one to a rate R 1 with the decoder having access to encoded side information Y , compressed at rate R 2 by encoder two, as well as the compressed version of X from the first encoder.The SCSI scenario arises in a variety of applications. For example, in video applications [1] X can represent a current frame, and Y a separate correlated frame sent from a second encoder. Y can even represent the frame(s) preceding the current frame X in the stream: while the previous frames are certainly available to the encoder, the encoder's coding scheme can be simplified by not making use of this information and leaving the decoder to exploit the interframe dependence. A second example can be found in communication in networks with relays [2]. A source sends a message X to a sink in a network containing a relay. One mode of operation for the relay is "compress and forward", i.e. for the relay to send a compressed version of its observation, Y , of the source-sink message to the sink. This compressed message can be used by the sink to further aid its decoding. SCSI appears in applications even beyond communication, for example (with minor changes) it has been proposed as a model for rate-constrained pattern recognition [3].For the lossless problem with coded side information (SCCSI) 1 , and the lossy problem with full side information (Wyner-Ziv), the "rate region" problem, i.e. determining the rates required to meet a given average distortion constraint, is solved. In this paper, we study these two problems from an error-exponent standpoint. Our motivation for doing so is three-fold:

Second-Order and Moderate Deviation Asymptotics for Successive Refinement

“…To subsequently apply this lemma to calculate the joint excess-distortion probability, we need to define the notion of Gaussian types (cf. [16], [17]) carefully. Indeed, the quantizations of the power of the source for the second-order and moderate deviations analyses are different and they need to be chosen carefully.…”

Nonasymptotic and Second-Order Achievability Bounds for Coding With Side-Information

“…Also letZ =Ỹ per (172). It should be noted here that, in this case, T c (γ c ) defined in (173) is equivalent to T WAK c (γ c ) defined in (51). Now, let us consider the stochastic map ϕ C constructed from the smoothed measureP LỸ (cf.…”

Error exponents for the relay channel

“…For PDF, our work leverages on the proofs of the various forms of the packing lemmas for multiuser channels in Haroutunian et al [9]. For CF, since it is related to Wyner-Ziv, we leverage on the work of Kelly-Wagner [10] who derived an achievable exponent for Wyner-Ziv. In a similar vein, [11] and [12] derived lower bounds for the error exponents of Gel'fand-Pinsker and content identification respectively.…”

“…The receiver decodes both the bin index and the message. This combination of covering and packing leads to a more involved analysis that needs to leverage on ideas in [10] where the error exponent for Wyner-Ziv was derived. It also leverages on a technique [6] to analyze the error when two indices are to be simultaneously decoded given a channel output.…”

“…This is analogous to maximum-likelihood decoding [15]. See (10). Define the set of conditional types…”