Reliability function for streaming over a DMC with feedback

Abstract

Conventionally, posterior matching is investigated in channel coding and block encoding contexts -the source symbols are equiprobably distributed and are entirely known by the encoder before the transmission. In this paper, we consider a streaming source, whose symbols progressively arrive at the encoder at a sequence of deterministic times. We derive the joint source-channel coding (JSCC) reliability function for streaming over a discrete memoryless channel (DMC) with feedback. We propose a novel instantaneous encoding phase that operates during the symbol arriving period and that achieves the JSCC reliability function for streaming when followed by a block encoding scheme that achieves the JSCC reliability function for a classical source whose symbols are fully accessible before the transmission. During the instantaneous encoding phase, the evolving message alphabet is partitioned into groups whose priors are close to the capacity-achieving distribution, and the encoder determines the group index of the actual sequence of symbols arrived so far and applies randomization to exactly match the distribution of the transmitted index to the capacity-achieving one. Surprisingly, the JSCC reliability function for streaming is equal to that for a fully accessible source, implying that the knowledge of the entire symbol sequence before the transmission offers no advantage in terms of the reliability function. For streaming over a symmetric binary-input DMC, we propose a one-phase instantaneous small-enough difference (SED) code that not only achieves the JSCC reliability function, but also, thanks to its single-phase time-invariant coding rule, can be used to stabilize an unstable linear system over a noisy channel. For equiprobably distributed source symbols, we design low complexity algorithms to implement both the instantaneous encoding phase and the instantaneous SED code. The algorithms group the source sequences into sets we call types, which enable the encoder and the decoder to track the priors and the posteriors of source sequences jointly, leading to a log-linear complexity in time. While the reliability function is derived for non-degenerate DMCs, i.e., DMCs whose transition probability matrix has all positive entries, for degenerate DMCs we design a code with instantaneous encoding that achieves zero error for all rates below Shannon's joint source-channel coding limit.

Reliability Function for Streaming Over a DMC With Feedback

“…We call the group partitioning rule in ( 43)-( 44) instantaneous small-enough difference (SED) rule since it reduces to Naghshvar et al's SED rule [8] if the source is fully accessible to the encoder before the transmission. The rule ensures that the difference between a group prior π x (y t−1 ) and its corresponding capacity-achieving probability P * X (x) = 1 2 , x ∈ {0, 1} is bounded by the source prior on the right side of (44).…”

“…First, in step (iii ) (group partitioning), we use the approximating instantaneous SED rule to mimic the exact rule in ( 43)- (44). The minimum of the objective function in (53a) is equal to the difference |π 0 (y t−1 ) − π 1 (y t−1 )| between the group priors of the partition {G x (y t−1 )} x∈{0,1} obtained by the approximating rule in step (iii ).…”

“…If π 0 (y t−1 ) ≥ π 1 (y t−1 ), (54) recovers ( 44) since θ Sj (y t−1 ) is the smallest prior in G 0 (y t−1 ), thus the approximating instantaneous SED rule recovers the exact rule. If π 0 (y t−1 ) < π 1 (y t−1 ), θ Sj (y t−1 ) on the right side of (54) is the largest prior in G 1 (y t−1 ), violating the right side of (44).…”

“…A part of this work is presented at the 2022 IEEE International Symposium on Information Theory [44]. The conference version does not contain Sections V-VIII or any proofs.…”