An efficient feedback coding scheme with low error probability for discrete memoryless channels

In this paper, a general binary-input binary-output (BIBO) channel is investigated in the presence of feedback and input constraints. The feedback capacity and the optimal input distribution of this setting are calculated for the case of an (1, ∞)-RLL input constraint, that is, the input sequence contains no consecutive ones. These results are obtained via explicit solution of an equivalent dynamic programming optimization problem. A simple coding scheme is designed based on the principle of posterior matching, which was introduced by Shayevitz and Feder for memoryless channels. The posterior matching scheme for our input-constrained setting is shown to achieve capacity using two new ideas: history bits, which captures the memory embedded in our setting, and message-interval splitting, which eases the analysis of the scheme. Additionally, in the special case of an S-channel, we give a very simple zero-error coding scheme that is shown to achieve capacity. For the input-constrained BSC, we show using our capacity formula that feedback increases capacity when the cross-over probability is small.

Section: ) Encoder Operation At Time Imentioning

confidence: 93%

“…In any case, note that we always have x − i+1 (J i+1 ) = x i . The case when J has three children is similarly handled, based on (24).…”

Section: A) Cyclic Shifting and Message-interval Splittingmentioning

confidence: 99%

Feedback Capacity and Coding for the BIBO Channel With a No-Repeated-Ones Input Constraint

Sabag

Permuter

Kashyap

2018

“…Note that for V n = 0 this coincides with the classical PM scheme [1]. The randomization idea is key to our simplified analysis, and is due to Li and El Gamal [8] who analyzed a non-sequential fixed-rate fixed-block-length version of this scheme in a DMC setting.…”

Section: Randomized Posterior Matchingmentioning

confidence: 95%

“…A key ingredient in their scheme was a random shift applied to the message point after each PM iteration, which circumvented some of the analysis obstacles. Both [7] and [8] also provide error exponent results.…”

Section: Introductionmentioning

confidence: 99%

A Simple Proof for the Optimality of Randomized Posterior Matching

Shayevitz

Feder

2016

Posterior matching (PM) is a sequential horizon-free feedback communication scheme introduced by the authors, who also provided a rather involved optimality proof showing it achieves capacity for a large class of memoryless channels. Naghshvar et al considered a non-sequential variation of PM with a fixed number of messages and a random decision-time, and gave a simpler proof establishing its optimality via a novel Shannon-Jensen divergence argument. Another simpler optimality proof was given by Li and El Gamal, who considered a fixed-rate fixed block-length variation of PM with an additional randomization. Both these works also provided error exponent bounds. However, their simpler achievability proofs apply only to discrete memoryless channels, and are restricted to a non-sequential setup with a fixed number of messages. In this paper, we provide a short and transparent proof for the optimality of the fully sequential horizon-free PM scheme over general memoryless channels. Borrowing the key randomization idea of Li and El Gamal, our proof is based on analyzing the random walk behavior of the shrinking posterior intervals induced by a reversed iterated function system (RIFS) decoder.

“…In a fixed-length setting, this simple one-phase scheme is known to achieve the capacity of a BSC [8], and its posterior matching extension has recently been shown to achieve the capacity of general DMCs with noiseless feedback [9]. Li and El Gamal [10] proposed a variant of the posterior matching scheme and derived a lower bound on its error exponent in the fixed-length setting.…”

Section: Introductionmentioning

confidence: 99%

Extrinsic Jensen–Shannon Divergence: Applications to Variable-Length Coding

Naghshvar

Javidi

Wigger

2015

112

Abstract-This paper considers the problem of variable-length coding over a discrete memoryless channel (DMC) with noiseless feedback. The paper provides a stochastic control view of the problem whose solution is analyzed via a newly proposed symmetrized divergence, termed extrinsic Jensen-Shannon (EJS) divergence. It is shown that strictly positive lower bounds on EJS divergence provide non-asymptotic upper bounds on the expected code length. The paper presents strictly positive lower bounds on EJS divergence, and hence non-asymptotic upper bounds on the expected code length, for the following two coding schemes: variable-length posterior matching and MaxEJS coding scheme which is based on a greedy maximization of the EJS divergence.As an asymptotic corollary of the main results, this paper also provides a rate-reliability test. Variable-length coding schemes that satisfy the condition(s) of the test for parameters R and E, are guaranteed to achieve rate R and error exponent E. The results are specialized for posterior matching and MaxEJS to obtain deterministic one-phase coding schemes achieving capacity and optimal error exponent. For the special case of symmetric binary-input channels, simpler deterministic schemes of optimal performance are proposed and analyzed.Index Terms-Discrete memoryless channel, variable-length coding, sequential analysis, feedback gain, Burnashev's reliability function, optimal error exponent.