Mutual information of the white Gaussian channel with and without feedback

On the connections between information and estimation theory: from AWGN Channels to dynamical systems with feedback

“…The result generalizes the classical Duncan's theorem for AWGNCs with or without feedback [1], [23]. It also provides a general condition guaranteeing the fulfilment of requirement (14) in Definition 2.1.…”

“…As mentioned before, Theorem 3.1, which relates inputoutput mutual information (I) and CMMSE (cmmse φ ) for the general input-output dynamical system (6), generalizes the classical Duncan's theorem for AWGNCs with or without feedback [1], [23]. Indeed, for the AWGNC with feedback we have G ≡ 1, therefore φ ≡ F , and hence equation (16) in Theorem 3.1 reduces to…”

“…Relationship (16) had been previously established in the context of AWGNCs (with or without feedback [1], [23]), and under condition (17).…”

“…Specifically, it is shown that an analogous relationship to (2) can be written in this setting, so extending classical Duncan's theorem for standard additive white Gaussian noise channels with and without feedback [1], [23] to this generalized model. Proofs are in the framework of absolutely continuity properties of stochastic process measures 5 , subjacent to the Girsanov's theorem [3], [25].…”

A View of Information-Estimation Relations in Gaussian Networks

“…Information theoretic measure can indeed be put in terms of log-likelihood ratios, however, these works did not make this additional connecting step. In the early 1970s continuous-time signals observed in white Gaussian noise received specific attention in the work of Duncan [4] and Kadota et al [5] who investigated connections between the mutual information and causal filtering. In particular, Duncan and Zakai (Duncan's theorem was independently obtained by Zakai in the general setting of inputs that may depend causally on the noisy output in a 1969 unpublished Bell Labs Memorandum (see [6])) [4,7] showed that the input-output mutual information can be expressed as a time integral of the causal minimum mean square error (MMSE).…”