Improved Upper Bounds to the Causal Quadratic Rate-Distortion Function for Gaussian Stationary Sources

Abstract

We improve the existing achievable rate regions for causal and for zero-delay source coding of stationary Gaussian sources under an average mean squared error (MSE) distortion measure. To begin with, we find a closed-form expression for the information-theoretic causal rate-distortion function (RDF) under such distortion measure, denoted by R , where R(D) denotes Shannon's RDF. Two of these bounds are strictly smaller than 0.5 bits/sample at all rates. These bounds differ from one another in their tightness and ease of evaluation; the tighter the bound, the more involved its evaluation.We then show that, for any source spectral density and any positive distortion D ≤ σ 2 x , R it c (D) can be realized by an AWGN channel surrounded by a unique set of causal pre-, post-, and feedback filters. We show that finding such filters constitutes a convex optimization problem. In order to solve the latter, we propose an iterative optimization procedure that yields the optimal filters and is guaranteed to converge to R it c (D). Finally, by establishing a connection to feedback quantization we design a causal and a zerodelay coding scheme which, for Gaussian sources, achieves an operational rate lower than R it c (D)+0.254

Rate-Cost Tradeoffs in Control

“…Causal rate-distortion function is challenging to evaluate, and beyond the scalar Gauss-Markov source [2], [23], no closed-form expression is known for it. For stationary scalar Gaussian processes, Derpich and Ostergaard [32] showed an upper bound and Silva et al [29] a lower bound. For vector Gauss-Markov sources, Tanaka et al developed a semidefinite program to compute exactly the minimum directed mutual information in quantization [33] and control [34].…”

“…For the scalar Gaussian system, (16) holds with equality. This is a consequence of known analyses [23], [2], [32,Th. 3] (see also Remarks 5 and 6 in Section III below).…”

Fixed-Rate Zero-Delay Source Coding for Stationary Vector-Valued Gauss-Markov Sources

“…After the definition, we explain the optimal test-channel that corresponds to this Gaussian source model [7]. First, notice that for general continuous alphabet sources, i.e., sources that are [5,6]), where R(D) denotes the classical RDF [10]. Note that, inequality (a) is strict, in general, and becomes equality when the source is i.i.d.…”

“…In this paper, we consider a fixed-rate zero-delay source coding problem where a vector-valued Gaussian source modeled as a stationary linear time-invariant (LTI) vector-valued Gauss-Markov process is compressed subject to a MSE distortion constraint. We tackle the problem, by considering the NRDF [4] which is known to be a tighter lower bound to the zero-delay RDF compared to the classical RDF (for details see, e.g., [5,6]). We use the optimal test-channel that corresponds to the NRDF of the aforementioned stationary Gaussian source model under the MSE distortion constraint, to characterize the stationary Gaussian NRDF and evaluate its corresponding information rates [6,7].…”

Asymptotic Reverse-Waterfilling Characterization of Nonanticipative Rate Distortion Function of Vector-Valued Gauss-Markov Processes with MSE Distortion

“…Tatikonda et al in [3] revisited the nonanticipatory epsilon entropy, for timeinvariant scalar and vector-valued Gauss-Markov processes subject to pointwise MSE distortion function, under the name "sequential RDF", and identified connections between unstable eignevalues of linear Gaussian control systems and the minimum rate requirements to stabilize such systems, when feedback is applied through a limited rate channel (memoryless). Derpich and Østergaard in [4] characterized variants of the nonanticipatory −entropy for stationary scalar-valued Gaussian autoregressive models with pointwise MSE distortion function. Tanaka et al in [5] revisited the sequential RDF of a vector-valued Gauss-Markov process subject to a pointwise MSE distortion function and applied semidefinite programming (under certain assumptions) to compute its optimal value numerically.…”