Efficient universal noiseless source codes

Abstract

Abstract-Although the existence of universal noiseless variable-rate codes for the class of discrete stationary ergodic sources has previously been established, very few practical universal encoding methods are available. Efficient implementable universal source coding techniques are discussed in this paper. Results are presented on source codes for which a small value of the maximum redundancy is achieved with a relatively short block length. A constructive proof of the existence of universal noiseless codes for discrete stationary sources is first presented. The proof is shown to provide a method for obtaining efficient universal noiseless variable-rate codes for various classes of sources. For memoryless sources, upper and lower bounds are obtained for the minimax redundancy as a function of the block length of the code. Several techniques for constructing universal noiseless source codes for memoryless sources are presented and their redundancies are compared with the bounds. Consideration is given to possible applications to data compression for certain nonstationary sources.

Efficient compression of monotone and m-modal distributions

“…Let I n k be the class of all i.i.d. distributions over k elements and block length n. Redundancy of I n k has been extensively studied [4][5][6][7][8][9][10][11]. It is now well established that 1) For k = o(n)…”

Number of hidden states and memory: a joint order estimation problem for Markov chains with Markov regime

“…Lemma 3.4 is a combination of results which essentially go back to [23] and [6]. The proof is similar to the proof of [16], Lemma 3.4 and thus omitted.…”

“…Let us fix (k, m) ∈ N × N and denote by Q k,m the coding probability KT k,m or NML k,m . Let us recall that τ is defined by (6). Then the following bounds hold:…”

An Integral Representation of the Logarithmic Function with Applications in Information Theory

“…for a certain non-negative function g and positive constant q such that 8 We refer to this kind of density as a generalized multivariate Cauchy density, because the multivariate Cauchy density is obtained as a special case where g(x) = x 2 and q = 1 2 (n + 1). Using the Laplace transform relation,…”