Instantaneous SED coding over a DMC

Guo, Nian; Kostina, Victoria

doi:10.1109/isit45174.2021.9518087

“…The decoder forms the estimate B at time λ n by choosing the binary string i * that achieves the maximum in the right side of (16). The stopping rule (16) ensures that the error probability in (6) does not exceed , see Appendix A for details.…”

Section: A the Instantaneous Sed Code: Algorithmmentioning

confidence: 99%

Instantaneous SED coding over a DMC

Guo¹,

Kostina²

2021

Preprint

Self Cite

0

View full text Add to dashboard Cite

In this paper, we propose a novel code for transmitting a sequence of n message bits in real time over a discretememoryless channel (DMC) with noiseless feedback, where the message bits stream into the encoder one by one at random time instants. Similar to existing posterior matching schemes with block encoding, the encoder in our work takes advantage of the channel feedback to form channel inputs that contain the information the decoder does not yet have, and that are distributed close to the capacity-achieving input distribution, but dissimilar to the existing posterior matching schemes, the encoder performs instantaneous encoding -it immediately weaves the new message bits into a continuing transmission. A posterior matching scheme by Naghshvar et al. partitions the source messages into groups so that the group posteriors have a small-enough difference (SED) to the capacity-achieving distribution, and transmits the group index that contains the actual message. Our code adopts the SED rule to apply to the evolving message alphabet that contains all the possible variable-length strings that the source could have emitted up to that time. Our instantaneous SED code achieves better delay-reliability tradeoffs than existing feedback codes over 2-input DMCs: we establish this dominance both by simulations and via an analysis comparing the performance of the instantaneous SED code to Burnashev's reliability function.Due to the message alphabet that grows exponentially with time t, the complexity of the instantaneous SED code is doubleexponential in t. To overcome this complexity barrier to practical implementation, we design a low-complexity code for binary symmetric channels that we name the instantaneous type set SED code. It groups the message strings into sets we call type sets and tracks their prior and posterior probabilities jointly, resulting in the reduction of complexity from double-exponential to O(t 4 ). Simulation results show that the gap in performance between the instantaneous SED code and the instantaneous type-set SED code is negligible.

show abstract

“…Intuitively, the buffer-then-transmit code is a good choice if the buffering time is negligibly short, i.e., if data packets arrive at the encoder at an extremely fast rate. However, if data packets arrive at the encoder steadily rather than in a burst, the buffer-then-transmit code becomes ill-suited due to the delay introduced by collecting data into a block before the transmission [11]. The encoder in this paper performs instantaneous encoding: it starts transmitting as soon as the first message symbol arrives and incorporates new message symbols into the continuing transmission on the fly.…”

Section: Introductionmentioning

confidence: 99%

Reliability Function for Streaming Over a DMC With Feedback

Guo

¹

,

Kostina

²

2023

IEEE Trans. Inform. Theory

3

0

View full text Add to dashboard Cite

Conventionally, posterior matching is investigated in channel coding and block encoding contexts -the source symbols are equiprobably distributed and are entirely known by the encoder before the transmission. In this paper, we consider a streaming source, whose symbols progressively arrive at the encoder at a sequence of deterministic times. We derive the joint source-channel coding (JSCC) reliability function for streaming over a discrete memoryless channel (DMC) with feedback. We propose a novel instantaneous encoding phase that operates during the symbol arriving period and that achieves the JSCC reliability function for streaming when followed by a block encoding scheme that achieves the JSCC reliability function for a classical source whose symbols are fully accessible before the transmission. During the instantaneous encoding phase, the evolving message alphabet is partitioned into groups whose priors are close to the capacity-achieving distribution, and the encoder determines the group index of the actual sequence of symbols arrived so far and applies randomization to exactly match the distribution of the transmitted index to the capacity-achieving one. Surprisingly, the JSCC reliability function for streaming is equal to that for a fully accessible source, implying that the knowledge of the entire symbol sequence before the transmission offers no advantage in terms of the reliability function. For streaming over a symmetric binary-input DMC, we propose a one-phase instantaneous small-enough difference (SED) code that not only achieves the JSCC reliability function, but also, thanks to its single-phase time-invariant coding rule, can be used to stabilize an unstable linear system over a noisy channel. For equiprobably distributed source symbols, we design low complexity algorithms to implement both the instantaneous encoding phase and the instantaneous SED code. The algorithms group the source sequences into sets we call types, which enable the encoder and the decoder to track the priors and the posteriors of source sequences jointly, leading to a log-linear complexity in time. While the reliability function is derived for non-degenerate DMCs, i.e., DMCs whose transition probability matrix has all positive entries, for degenerate DMCs we design a code with instantaneous encoding that achieves zero error for all rates below Shannon's joint source-channel coding limit.

show abstract

“…With the emergence of the Internet of Things (IoT), communication systems, such as those employed in distributed control and tracking scenarios, are becoming increasingly dynamic, interactive, and delay-sensitive. Block encoding schemes are ill-suited to the streaming source context due to the delay induced by collecting message symbols into a block before the transmission [21]. The encoder in this paper performs N. Guo and V. Kostina are with the Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, 91125 USA.…”

Section: Introductionmentioning

confidence: 99%

“…While most feedback coding schemes in the literature considered block encoding of a source whose outputs are accessible in their entirety before the transmission, [2]- [12], [16], several existing works considered instantaneous encoding of a streaming source [18]- [21]. Chang and Sahai [18] considered instantaneous encoding for the transmission over a binary erasure channel with feedback of i.i.d.…”

Section: Introductionmentioning

confidence: 99%

“…Antonini et al [20] designed a causal encoding scheme for streaming bits with a fixed arrival rate over a BSC and showed by simulation that the code rate approaches the channel capacity as the bit arrival rate approaches the transmission rate. In our previous work [21], we proposed a code that uses an adapted SED rule [10] to instantaneously transmit randomly arriving bits and that leads to an achievability bound on the reliability function for binary-input DMCs with instantaneous encoding, and designed a polynomial-time version of it, applicable to transmitting Bernoulli- 1 2 bits over a BSC. In this paper, we propose a novel coding phase -the instantaneous encoding phase -for transmitting a sequence of k source symbols over a DMC with feedback.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reliability function for streaming over a DMC with feedback

Guo¹,

Kostina²

2022

Preprint

Self Cite

0

View full text Add to dashboard Cite

Conventionally, posterior matching is investigated in channel coding and block encoding contexts -the source symbols are equiprobably distributed and are entirely known by the encoder before the transmission. In this paper, we consider a streaming source, whose symbols progressively arrive at the encoder at a sequence of deterministic times. We derive the joint sourcechannel coding (JSCC) reliability function for streaming over a discrete memoryless channel (DMC) with feedback under regularity conditions. We propose a novel instantaneous encoding phase that operates during the symbol arriving period and that achieves the JSCC reliability function for streaming when followed by a block code that achieves the JSCC reliability function for a classical source whose symbols are fully accessible before the transmission. The instantaneous encoding phase partitions the evolving message alphabet into groups whose priors are close to the capacity-achieving distribution, and randomizes the group indices to ensure that the transmitted group index has the capacity-achieving distribution. Surprisingly, the JSCC reliability function for streaming is equal to that for a fully accessible source, implying that the knowledge of the entire symbol sequence before the transmission offers no advantage in terms of the reliability function. For equiprobably distributed source symbols, we design a low complexity algorithm for the instantaneous encoding phase. It groups the source sequences into sets we call types, which enable the encoder and the decoder to track the priors and the posteriors of source sequences jointly, leading to a log-linear complexity in time.

show abstract

Instantaneous SED coding over a DMC

Cited by 8 publications

References 15 publications

Instantaneous SED coding over a DMC

Instantaneous SED coding over a DMC

Reliability Function for Streaming Over a DMC With Feedback

Reliability function for streaming over a DMC with feedback

Contact Info

Product

Resources

About