Improved upper bounds to the causal quadratic rate-distortion function for Gaussian stationary sources

Abstract

We improve the existing achievable rate regions for causal and for zero-delay source coding of stationary Gaussian sources under an average mean squared error (MSE) distortion measure. To begin with, we find a closed-form expression for the information-theoretic causal rate-distortion function ( , where R(D) denotes Shannon's RDF. Two of these bounds are strictly smaller than 0.5 bits/sample at all rates. These bounds differ from one another in their tightness and ease of evaluation; the tighter the bound, the more involved its evaluation.We then show that, for any source spectral density and any positive distortion D ≤ σ 2 x , R it c (D) can be realized by an AWGN channel surrounded by a unique set of causal pre-, post-, and feedback filters. We show that finding such filters constitutes a convex optimization problem. In order to solve the latter, we propose an iterative optimization procedure that yields the optimal filters and is guaranteed to converge to R it c (D). Finally, by establishing a connection to feedback quantization we design a causal and a zerodelay coding scheme which, for Gaussian sources, achieves an operational rate lower than R it c (D)+0.254 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.

Performance analysis and optimal filter design for sigma-delta modulation via duality with DPCM

“…The optimal sigma-delta filter design problem was studied by several authors, under various assumptions [1], [6], [8], [12]- [15]. However, to the best of our knowledge, the simple expression from Theorem 2 for the optimal filter as the optimal predictor of S n from the past of {S n + W n } is novel.…”

“…However, to the best of our knowledge, the simple expression from Theorem 2 for the optimal filter as the optimal predictor of S n from the past of {S n + W n } is novel. The references most relevant to Theorem 2, are perhaps [14] and [8], [15]. In [14], Spang and Schultheiss formulated an optimization problem for finding the best FIR filter with p coefficients in a sigma-delta modulator with a scalar quantizer, under a fixed overload probability.…”

“…Their optimization problem can be solved numerically, but no closed form solution was given. In [8] and [15] the design of an optimal unconstrained sigma-delta filter was studied, under the assumption of a fixed scalar quantizer which can only be scaled in order to control the overload probability. Equations that characterize the optimal filter were derived.…”

Conditional downsampling for energy-efficient communications in wireless sensor networks

“…The work in [8] extends the general results of [7] for the case where side information, i.e., extra information that is correlated with the source, is available at the encoder and the decoder. In addition, a causal source code is called a zero-delay or sequential code if both the encoder and the decoder are causal (note that for the causal source code definition, the assumption of causality is only at the decoder) [9,10].…”

“…The authors in [11] provide analytical results for the existing duality between the rate distortion of an AR process with the capacity of the inter-symbol interference channels. By contrast, other works such as [9] follow an information theoretical approach that adjusts the upper and lower limits of the rate distortion for generic zero-delay schemes using the mutual information as a measure of the achievable rate.…”