Finite-Length Analyses for Source and Channel Coding on Markov Chains

Abstract

We study finite-length bounds for source coding with side information for Markov sources and channel coding for channels with conditional Markovian additive noise. For this purpose, we propose two criteria for finite-length bounds. One is the asymptotic optimality and the other is the efficient computability of the bound. Then, we derive finite-length upper and lower bounds for coding length in both settings so that their computational complexity is efficient. To discuss the first criterion, we derive the large deviation bounds, the moderate deviation bounds, and second order bounds for these two topics, and show that these finite-length bounds achieves the asymptotic optimality in these senses. For this discussion, we introduce several kinds of information measure for transition matrices.

Asymptotic and non-asymptotic analysis for a hidden Markovian process with a quantum hidden system

“…Hence, the outcome has long-period memory. When discussing some information theoretic problem, we need to discuss information theoretical quantity, e.g., entropy and conditional entropy instead of the sample mean [12]. In this case, we need to be careful with such a memory.…”

“…While in the non-asymptotic analysis, we derive upper and lower bounds for the tail probability, we need to consider requirements for a good bound because we need to distinguish good bounds from trivial bounds. Similarly to [12], we impose the following requirements on good bounds.…”

Dense Coding with Locality Restriction for Decoder: Quantum Encoders vs. Super-Quantum Encoders

“…This class of classical channels is often called generalized additive [79, Section V] or conditional additive [79,Section 4] and contains a class of additive channels as a subclass. Such a channel appears even in wireless communication by considering binary phase-shift keying (BPSK) modulations [80,Section 4.3]. Its most simple example is the binary symmetric channel (BSC).…”

“…The reference [81, Section VII-A-2] studied its quantum extension with an additive group, and discussed the capacity and the wire-tap capacity with the semantic security. Since this class has a good symmetric property, algebraic codes achieve the capacity [78,[80][81][82][83]. Since a algebraic code has less calculation complexity in comparison with other types of codes, this fact shows the usefulness of this class of classical channels.…”