Universal lossless compression via multilevel pattern matching

Abstract

A universal lossless data compression code called the multilevel pattern matching code (MPM code) is introduced. In processing a finite-alphabet data string of length , the MPM code operates at (log log) levels sequentially. At each level, the MPM code detects matching patterns in the input data string (substrings of the data appearing in two or more nonoverlapping positions). The matching patterns detected at each level are of a fixed length which decreases by a constant factor from level to level, until this fixed length becomes one at the final level. The MPM code represents information about the matching patterns at each level as a string of tokens, with each token string encoded by an arithmetic encoder. From the concatenated encoded token strings, the decoder can reconstruct the data string via several rounds of parallel substitutions. A (1 log) maximal redundancy/sample upper bound is established for the MPM code with respect to any class of finite state sources of uniformly bounded complexity. We also show that the MPM code is of linear complexity in terms of time and space requirements. The results of some MPM code compression experiments are reported.

A Space-Saving Approximation Algorithm for Grammar-Based Compression

“…LZW [21] including LZ78 [24], and BISECTION [8] are considered as algorithms that computes straight-line programs, CFGs formed from Chomsky normal form formulas. Also algorithms for restricted CFGs have been presented in [6], [10], [15], [16], [22].…”

An Efficient Pattern Matching Algorithm on a Subclass of Context Free Grammars

“…MPM code was developed by Kieffer et al [11] for efficient grammar-based text compression. Since MPM grammar sizes can be exponentially small with respect to the original string sizes, it is a rather hard task to solve the pattern matching problem in time proportional only to the grammar sizes.…”

“…Since computing a minimal SLP that generates a given string is known to be NP-complete, it is of great significance to develop approximative algorithms for generating small grammars [19,2]. One of those algorithms is the multilevel pattern matching code (MPM code) introduced by Kieffer et al [11]. MPM code is attractive in that it performs in linear time with respect to the input string size, and is capable of exponential compression -the generated grammar size can be exponentially small with respect to the input string size.…”

XML compression techniques: A survey and comparison

“…Following the idea of XMill, AXECHOP divides the source XML document into structural and data parts. However, for the structure part, AXECHOP uses a byte tokenization scheme that preserves the original structure of the document and then uses the MPM compression algorithm [33] to generate a context-free grammar which is represents this structure information. The generated grammar is then passed through an adaptive arithmetic coder before being written to the compressed file.…”