Optimal Estimation via Nonanticipative Rate Distortion Function and Applications to Time-Varying Gauss--Markov Processes

Abstract

In this paper, we develop finite-time horizon causal filters using the nonanticipative rate distortion theory. We apply the developed theory to design optimal filters for time-varying multidimensional Gauss-Markov processes, subject to a mean square error fidelity constraint. We show that such filters are equivalent to the design of an optimal {encoder, channel, decoder}, which ensures that the error satisfies a fidelity constraint. Moreover, we derive a universal lower bound on the mean square error of any estimator of time-varying multidimensional Gauss-Markov processes in terms of conditional mutual information. Unlike classical Kalman filters, the filter developed is characterized by a reverse-waterfilling algorithm, which ensures that the fidelity constraint is satisfied. The theoretical results are demonstrated via illustrative examples.

Asymptotic Reverse-Waterfilling Characterization of Nonanticipative Rate Distortion Function of Vector-Valued Gauss-Markov Processes with MSE Distortion

“…It is well-known that the optimization problem of (7) is convex with respect to the set of test channels P(dy n ||x n ) that satisfy the average (over time) MSE distortion, for D ∈ (0, D max ) ⊆ (0, ∞), and there exists an optimal solution characterizing it under general source distributions and distortion functions (e.g., [10]). By [8,Theorem 4.1], the optimal "test channel" corresponding to R 0,n (D) is of the form…”

“…Note that by choosing ∆ as in (32), we ensure that ∆ Λ. We wish to remark that the extension of Theorem 3 and Proposition 1 to the non-asymptotic regime appears in [8].…”

“…Recently, Stavrou et al in [6] characterized the nonanticipatory −entropy under the name nonanticipative rate distortion function (NRDF), for a time-varying scalar-valued Gauss-Markov process subject to a total (or average in time) MSE distortion function and characterized it via the solution of a reverse-waterfilling problem, similar to the well-known reverse-waterfilling of classical RDF [7]. The extension of [6] to time-varying R p -valued Gauss-Markov processes is considered in [8].…”

“…[8, Theorem 5.3](Alternative characterization of (12)) The following alternative characterization of Definition 1, (5) holds. (a) The optimal reproduction distribution is realized by…”

Bounds on the Sum-Rate of MIMO Causal Source Coding Systems with Memory under Spatio-Temporal Distortion Constraints

“…This can be shown following, for instance, the techniques of [ 23 ]. By the structural properties of the test channel derived in ([ 24 ], Section 4), if the source is first-order Markov, i.e., with distribution , the test channel distribution is of the form . Finally, combining this structural result, with ([ 25 ], Theorem 1.8.6), it can be shown that if is Gaussian then a jointly Gaussian process achieves a smaller value of the information rates, and if is Gaussian and Markov, then the infimum in ( 26 ), ( 28 ) and ( 30 ) can be restricted to test channel distributions which are Gaussian, of the form .…”