The entropy of a code with probabilities

Rocha, Valdemar C. da; Oliveira, H. M. de

doi:10.1109/its.1998.718448

Cited by 2 publications

(1 citation statement)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Let lSI denote the number of states in a given Markov source. The probability Pu;, i = 1, 2, ... , lSI, of every state cr; is equal to the probability of the corresponding ttee node P; divided by the average codeword length [6].…”

Section: Rooted Trees and Markov Sourcesmentioning

confidence: 99%

Homophonic Sequence Substitucion

Rocha¹,

Oliveira²

1998

JCIS

View full text Add to dashboard Cite

Resumo • Substitui~ao homofOnica de sequencias e o nome dado neste trabalho para a tecnica que consiste em substituir um-a-um uma dada seqUencia de sfmbolos, finita ou semi-infinita, por outra seqUencia, respectivamente finita ou semi-infinita, sobre o mesmo alfabeto, porem com uma taxa de entropia mais elevada. A sequencia de safda de uma dada fonte discreta, estacionana e erg6dica, e codificada com urn c6digo de fonte binano sem perdas C. Uma concatenaao de palavras c6digo de C e entiio convenientemente segmentada e recodificada com urn c6digo de fonte binano sem perdas Iterando urn certo mlmero de vezes o ultimo passo descrito acima, prova-se que a taxa de entropia da seqUencia na safda do ultimo codificador aproxima-se do valor 1, assintoticamente, e portanto realizando a substitui~ao homofOnica 6tima A redundftncia remanescente, ap6s k codifi.ca~oes consecutivas, e 1-Hk ( S) bits por dfgito binano, onde Hk ( S) denota a taxa de entropia da sequencia resultante ap6s a kesima codifi.ca~ao. Urn modelo de fonte de Markov e apresentado para descrever as seqUencias binanas codifi.cadas e para computar as respectivas taxas de entropia.Abstract-Homophonic sequence substitution is the name given in this paper to the technique which consists of substituting one-to-one a given finite (or semi-infinite) sequence of symbols by another finite (or semi-infinite) sequence over the same alphabet but having a higher entropy rate. The output sequence of a given discrete stationary and ergodic source is encoded with a binary lossless source code C. A concatenation of codeworda of C is then conveniently parsed and reencoded with a binary lossless source code. By iterating the latter step a number of times, it is proved that the entropy rate of the binary sequence at the output of the last encoder approaches the value 1 asymptotically, therefore performing optimum homophonic sequence substitution. The remaining redundancy, after k consecutive encodings, is 1-Hk(S) bits per binary digit, where Hk ( S) is the entropy rate of the binary sequence resulting after the k'h encoding. A Markov source model is presented to describe the binary encoded sequences and to compute their entropy rate.

show abstract

Section: Rooted Trees and Markov Sourcesmentioning

confidence: 99%