Optimal non-prefix codeword sets

Abstract

We discuss the optimal variable-lengtlr-toblack encoding of a discrete memoryless source P without error. The process uses a codeword set W , vrich consist of n source blocks of variable length that are to . , l e encoded by a channel symbol.First we prove tight boun on the average coded length ac aveable by optimal prefix codeword sets. 9 or simplicity let we focus our attention only on the binary case. Let we denote the average coded lerqth of L(Wn), the entropy of the source by R(P)nr,d n ',tin by the redundancy of the cote by r (P) where = log2 n -.(p) LcvnfIf Vn is an optimal prefix codeword set for P and n then W e n P is not well balanced the constant c(F) can be large i.e. the average coded lengt I achieveable by prefix codeword sets ' ; i n is considerably worse than the theoretic bound.If we drop the prefix condition on Wn we can significally improve this. For non-prefix encoding we use the maximal snap principle, we encode the source se',uence by the longest codeword matching to it in \Vn. If P is almost concentrated on one point, the optimal non-prefix codeword set is actw ally a runlength code. Detailed CescriItion of the generation algorithm of <he optimal Wn Let w e call Jn a 'real non-prefix codeword set' if it is non-prefix, and not belongs to the runlength type mentioned above. There is only a mall territory in parameter space given by P = (p, 1-p) and n where the real Eon prefix codeword sets have advantages in c o ding : For every P there exist an index, tha% for every n ) E the optimal codeword set IVn is prefix. When the n p-ameter is comparably s m y , l l , sind P is not itell balanced, in most of the cases a non-prefix code of runlength type c%n achieve the optimum. Hard algorithmic difficulties arises i.hen vde want to generate Peal non-prefix codeword sets even when n is smell e.g. n = l o .A backward recursion algorithm on n starting from the o?timal W can produce a good solution. We discuss d e algorithm in detail m d estimate its efficiency. r n ( p ) = C ( P ) + o(1) and its redundancy is discussed. 234