Length of minimal forbidden words on a stationary ergodic source

Abstract

An antidictionary is in particular useful for data compression, and it consists of minimal forbidden words for a given string. We derive the average length M n of minimal forbidden words in strings of length n under a stationary ergodic source with entropy H which takes values on a finite alphabet. For the string length n, we prove, log ti]M« == H, in probability, as n -+ 00. We use the Wyner-Ziv result, with respect to connection between entropy and recurrence-time for ergodic processes, to prove the theorem. Its validity is shown by simulation results on a memoryless binary information source.