Pattern entropy - revisited

Abstract

Abstract-A pattern of a sequence is a sequence of integer indices with each index describing the order of first occurrence of the respective symbol in the original sequence. Several recent works studied entropy and entropy rate of patterns. Specifically, in a recent paper, tight general bounds on the block entropy of patterns of sequences generated by independent and identically distributed (i.i.d.) sources were derived. In this paper, precise approximations are given to the pattern block entropies for patterns of sequences generated by i.i.d. uniform and monotonic distributions, including distributions over the integers, and the geometric distribution. Numerical non-asymptotic bounds on the pattern block entropies of these distributions are provided even for very short blocks, and even for distributions that have infinite i.i.d. entropy rates. Conditional index entropy is also studied for distributions over smaller alphabets.