Codebook Cardinality Spectrum of Distributed Arithmetic Coding for Independent and Identically-Distributed Binary Sources

Abstract

It was demonstrated that, as a nonlinear implementation of Slepian-Wolf Coding (SWC), Distributed Arithmetic Coding (DAC) outperforms traditional Low-Density Parity-Check (LPDC) codes for short code length and biased sources. This fact triggers research efforts into theoretical analysis of DAC. In our previous work, we proposed two analytical tools, Codebook Cardinality Spectrum (CCS) and Hamming Distance Spectrum (HDS), to analyze DAC for Stationary Memoryless Binary Sources (SMBS) with uniform distribution. This paper extends our work on CCS from uniform SMBS to biased SMBS. We begin with the final CCS and then deduce each level of CCS backwards by recursion. The main finding of this paper is that the final CCS of biased SMBS is not uniformly distributed over [0, 1). This paper derives the final CCS of biased SMBS and proposes a numerical algorithm for calculating CCS effectively in practice. All theoretical analyses are well verified by experimental results.

Fair Numerical Algorithm of Coset Cardinality Spectrum for Distributed Arithmetic Coding

“…Hence, there is a rate loss of bits, where If we decode the bitstream along the path , then we will obtain a tuple of real numbers which can be deduced recursively. The forward recursion of can be easily transformed into an equivalent backward recursion as shown below [ 30 ]. The pdf of is called the i -th CCS and denoted by , , which is usually simplified as .…”

“…Then an important problem is how DAC coset cardinality is distributed, which can be answered by the so-called Coset Cardinality Spectrum (CCS). The idea of CCS budded in [ 27 , 28 ] and was formally defined for uniform and nonuniform binary sources in [ 29 , 30 ], respectively. The concept of CCS is very useful as it not only can serve as a theoretical tool to analyze the properties of DAC [ 31 , 32 ], but also can be used to derive correct decoding formulae [ 31 , 32 ] (See the discussion in the last paragraph of Section 2 ).…”

“…Instead, one can begin with the final pdf, which is usually simple and calculable, and then derive each pdf via a backward recursion. In [ 29 ], a numerical algorithm was proposed to implement the backward recursion for uniform binary sources, and then it was generalized to nonuniform binary sources in [ 30 ].…”

“…However, the numerical algorithm proposed in [ 29 , 30 ] is very primitive and lacks theoretical justification. This paper will make an in-depth analysis of the numerical algorithm proposed in [ 29 , 30 ] and observe the results carefully. It will be found that the numerical algorithm proposed in [ 29 , 30 ] does not exactly converge to the real CCS.…”

“…This paper will make an in-depth analysis of the numerical algorithm proposed in [ 29 , 30 ] and observe the results carefully. It will be found that the numerical algorithm proposed in [ 29 , 30 ] does not exactly converge to the real CCS. After a strict theoretical analysis, this paper will propose a novel numerical algorithm, which perfectly overcomes the drawbacks of the original numerical algorithm.…”