Extrinsic Jensen–Shannon Divergence: Applications to Variable-Length Coding

Abstract

Abstract-This paper considers the problem of variable-length coding over a discrete memoryless channel (DMC) with noiseless feedback. The paper provides a stochastic control view of the problem whose solution is analyzed via a newly proposed symmetrized divergence, termed extrinsic Jensen-Shannon (EJS) divergence. It is shown that strictly positive lower bounds on EJS divergence provide non-asymptotic upper bounds on the expected code length. The paper presents strictly positive lower bounds on EJS divergence, and hence non-asymptotic upper bounds on the expected code length, for the following two coding schemes: variable-length posterior matching and MaxEJS coding scheme which is based on a greedy maximization of the EJS divergence.As an asymptotic corollary of the main results, this paper also provides a rate-reliability test. Variable-length coding schemes that satisfy the condition(s) of the test for parameters R and E, are guaranteed to achieve rate R and error exponent E. The results are specialized for posterior matching and MaxEJS to obtain deterministic one-phase coding schemes achieving capacity and optimal error exponent. For the special case of symmetric binary-input channels, simpler deterministic schemes of optimal performance are proposed and analyzed.Index Terms-Discrete memoryless channel, variable-length coding, sequential analysis, feedback gain, Burnashev's reliability function, optimal error exponent.

Active Learning and CSI Acquisition for mmWave Initial Alignment

“…have been shown to be useful for analyzing adaptive systems [2], [29], [30], [31] with the Bayes' rule dynamics on the posterior distribution. Particularly, the functional average log-likelihood [30] has shown its usefulness in analyzing the behaviour of the posterior in feedback coding systems [28], dynamic spectrum sensing [10], hypothesis testing [32], active learning [32], etc. Here, we review some useful concepts through the context of AoA estimation with sequential beamforming:…”

“…Fact 2 (Lemma 2 in [30]). The EJS divergence is lower bounded by the Jensen Shannon (JS) divergence :…”

Moderate Deviation Asymptotics for Variable-Length Codes With Feedback

“…(18). However, we provide an alternative proof that holds for all T ∈ R which uses a different construction of a submartingale (compared to [26], [27]). We also show rigorously that τ is almost surely finite in Lemma 13, which is essential for the proof of Lemma 1 and of Lemmas 2 and 3 to follow.…”

Short-blocklength non-binary LDPC codes with feedback-dependent incremental transmissions

“…This is a form of active feedback in the sense that the feedback is telling the transmitter what to transmitter rather than telling the transmitter only whether additional bits from a pre-determined rate-compatible code family. This is a generalization of the ideas of active hypothesis testing [10]. For comparison, we also performed simulations using non-active feedback, in which the additional bits are selected at random.…”