In this paper we introduce a fundamental principle for optimal communication over general memoryless channels in the presence of noiseless feedback, termed posterior matching. Using this principle, we devise a (simple, sequential) generic feedback transmission scheme suitable for a large class of memoryless channels and input distributions, achieving any rate below the corresponding mutual information. This provides a unified framework for optimal feedback communication in which the Horstein scheme (BSC) and the Schalkwijk-Kailath scheme (AWGN channel) are special cases. Thus, as a corollary, we prove that the Horstein scheme indeed attains the BSC capacity, settling a longstanding conjecture. We further provide closed form expressions for the error probability of the scheme over a range of rates, and derive the achievable rates in a mismatch setting where the scheme is designed according to the wrong channel model. Several illustrative examples of the posterior matching scheme for specific channels are given, and the corresponding error probability expressions are evaluated. The proof techniques employed utilize novel relations between information rates and contraction properties of iterated function systems. . 1 The rate and error exponent analysis in the original papers [3, 6], while intuitively appealing, are widely considered to be non-rigorous. by U. A measurable bijective function µ : (0, 1) → (0, 1) is called a uniformity preserving function (u.p.f.) if Θ ∼ U implies that µ(Θ) ∼ U.A scalar distribution P X is said to be (strictly) dominated by another distribution P Y if F X (x) < F Y (x) whenever F Y (x) ∈ (0, 1), and the relation is denoted by P 0 for every A ∈ B, where B is the corresponding σ-algebra. This relation is denoted P X < < P Y . If both distributions are absolutely continuous w.r.t. each other, then they are said to be equivalent. The total variation distance between P X and P Y is defined asA statement is said to be satisfied for P X -a. a. (almost all) x, if the set of x's for which it is satisfied has probability one under P X .In what follows we use conv(·) for the convex hull operator, |∆| for the length of an interval ∆ ⊆ R, log for log 2 , range(f) for the range of a function f , and • for function composition. The indicator function over a set A is denoted by 1 A (·). A set A ⊆ R m is said to be convex in the direction u ∈ R m , if the intersection of A with any line parallel to u is a connected set (possibly empty). Note that A is convex if and only if it is convex in any direction.The following simple lemma states that (up to discreteness issues) any real-valued r.v. can be shaped into a uniform r.v. or vice versa, by applying the corresponding c.d.f or its inverse, respectively. This fact is found very useful in the sequel Lemma II.
A coding scheme for the discrete memoryless broadcast channel with {noiseless, noisy, generalized} feedback is proposed, and the associated achievable region derived. The scheme is based on a block-Markov strategy combining the Marton scheme and a lossy version of the Gray-Wyner scheme with side-information. In each block the transmitter sends fresh data and update information that allows the receivers to improve the channel outputs observed in the previous block. For a generalization of Dueck's broadcast channel our scheme achieves the noiseless-feedback capacity, which is strictly larger than the no-feedback capacity. For a generalization of Blackwell's channel and when the feedback is noiseless our new scheme achieves rate points that are outside the no-feedback capacity region. It follows by a simple continuity argument that for both these channels and when the feedback noise is sufficiently low, our scheme improves on the no-feedback capacity even when the feedback is noisy.
We address the problem of universal communications over an unknown channel with an instantaneous noiseless feedback, and show how rates corresponding to the empirical behavior of the channel can be attained, although no rate can be guaranteed in advance. First, we consider a discrete moduloadditive channel with alphabet X , where the noise sequence Z n is arbitrary and unknown and may causally depend on the transmitted and received sequences and on the encoder's message, possibly in an adversarial fashion. Although the classical capacity of this channel is zero, we show that rates approaching the empirical capacity log |X | − Hemp(Z n ) can be universally attained, where Hemp(Z n ) is the empirical entropy of Z n . For the more general setting where the channel can map its input to an output in an arbitrary unknown fashion subject only to causality, we model the empirical channel actions as the modulo-addition of a realized noise sequence, and show that the same result applies if common randomness is available. The results are proved constructively, by providing a simple sequential transmission scheme approaching the empirical capacity. In part II of this work we demonstrate how even higher rates can be attained by using more elaborate models for channel actions, and by utilizing possible empirical dependencies in its behavior.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.