Metric and Topological Entropy Bounds for Optimal Coding of Stochastic Dynamical Systems

Abstract

We consider the problem of optimal zero-delay coding and estimation of a stochastic dynamical system over a noisy communication channel under three estimation criteria concerned with the low-distortion regime. The criteria considered are (i) a strong and (ii) a weak form of almost sure stability of the estimation error as well as (ii) quadratic stability in expectation. For all three objectives, we derive lower bounds on the smallest channel capacity C0 above which the objective can be achieved with an arbitrarily small error. We first obtain bounds through a dynamical systems approach by constructing an infinite-dimensional dynamical system and relating the capacity with the topological and the metric entropy of this dynamical system. We also consider information-theoretic and probabilitytheoretic approaches to address the different criteria. Finally, we prove that a memoryless noisy channel in general constitutes no obstruction to asymptotic almost sure state estimation with arbitrarily small errors, when there is no noise in the system. The results provide new solution methods for the criteria introduced (e.g., standard information-theoretic bounds cannot be applied for some of the criteria) and establish further connections between dynamical systems, networked control, and information theory, and especially in the context of nonlinear stochastic systems.f w (x) = f (x, w), f x (w) = f (x, w).so that f w : X → X and f x : W → X.

A Novelty in Blahut-Arimoto T ype Algorithms: Optimal Control over Noisy Commu nication Channels

“…Overview C Ontrol-theory with an integration of information theory has received brilliant attention in the recent decade. There arises a totally important question, in this context, how to optimally control the system over a noisy communication channel with feedback [1], [2], [3], [4], [5], [6], [7], [8]. Undoubtedly, the principle of distortion-function as the criterion of estimation-error 1 plays an entirely central role.…”

“…In relation to the information-constraint based issues in control theory, [4], [5], [6], [7], [8] are brilliant. Some probabilitytheoretic procedures as well as information-theoretic ones were applied in [4], [5].…”

“…In [6], invariance properties of nonlinear stochasticities for dynamical systems were taken into account with regard to information constraints. In [7], the problem of optimal zero-delay coding as well as the estimation of a stochastic dynamical system over a memoryless noisy communication-channel were explored. A lower-bound on the smallest channel capacity was obtained with an arbitrarily and small reconstruction-error 2 .…”

“…6 Which is a fundamental beneficiary compared to traditional gradient decent algorithms. 7 It achieves the optimal value via the local optima of the objective function successfully moving towards the global optimum. (a) General model of optimal control with feedback [1], [2], [3], [4], [5], [6], [7], [8].…”

A Novelty in Blahut-Arimoto T ype Algorithms: Optimal Control over Noisy Commu nication Channels

“…Overview C Ontrol-theory with an integration of information theory has received brilliant attention in the recent decade. There arises a totally important question, in this context, how to optimally control the system over a noisy communication channel with feedback [1], [2], [3], [4], [5], [6], [7], [8]. Undoubtedly, the principle of distortion-function as the criterion of estimation-error 1 plays an entirely central role.…”