On Optimal Coding of Non-Linear Dynamical Systems

Abstract

We consider the problem of zero-delay coding of a dynamical system over a discrete noiseless channel under three estimation criteria concerned with the low-distortion regime. For these three criteria, formulated stochastically in terms of a probability distribution for the initial state, we characterize the smallest channel capacities above which the estimation objectives can be achieved. The results establish further connections between topological and metric entropy of dynamical systems and information theory.

Metric and Topological Entropy Bounds for Optimal Coding of Stochastic Dynamical Systems

“…We also establish impossibility results when noise is such that the process is sufficiently mixing. We show that our results reduce to those reported in [16] for deterministic systems. An implication is that the rate bounds may not be continuously dependent on the presence of stochastic noise, i.e., an arbitrarily small noisy perturbation in the system dynamics may lead to a discontinuous change in the rate requirements for each of the criteria.…”

“…In this paper, we will provide further connections between the ergodic theory of dynamical systems and information theory by answering the problems posed in the previous section and relating the answers to the concepts of either metric or topological entropy. Our findings complement and generalize our results in [16] since here we consider stochasticity in the system dynamics and/or the communication channels.…”

“…In a recent work [16], we investigated the same problem for the special case involving only deterministic systems and discrete noiseless channels. In this paper, we will provide further connections between the ergodic theory of dynamical systems and information theory by answering the problems posed in the previous section and relating the answers to the concepts of either metric or topological entropy.…”

“…As we note in [16], optimal coding of stochastic processes is a problem that has been studied extensively; in information theory in the context of per-symbol cost minimization, in dynamical systems in the context of identifying representational and equivalence properties between dynamical systems, and in networked control in the context of identifying information transmission requirements for stochastic stability or cost minimization. As such, for the criteria laid out in (E1)-(E3) above, the results in our paper are related to the efforts in the literature in the following three general areas.…”

“…In our setup, we have causality as a restriction in coding and decoding. Zero-delay coding is an increasingly important research area of significant practical relevance, as we review in [16]. Notable papers include the classical works by Witsenhausen [43], Walrand & Varaiya [42] and Teneketzis [41].…”