Enhancements to Ziv-Lempel data compression

SIGARCH Comput. Archit. News

Wei

1991

Section: Move-to-front Codesmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

Systolic implementations of a move-to-front text compressor

SIGARCH Comput. Archit. News

Wei

1991

“…The Unix utility provides v e r y good compression (46-70% on our text files) at a moderately-high input bandwidth (30 Kbytes/second on a 1 MIPS machine such as a Sun-3). Recent research indicates that higher compression ratios, up to 83%, can be obtained either by the high-order Markov models of Cleary and Witten [3] and Rissanen [9], or by improved versions of 'compress' [10,5]. The high-order Markov models are hampered by their multi-megabyte *Supported by the National Science Foundation, through its Design, Tools and Test Program under grant number MIP 87-06139.…”

Section: Introductionmentioning

confidence: 99%

Systolic implementations of a move-to-front text compressor

Proceedings of the First Annual ACM Symposium on Parallel Algorithms and Architectures

Wei

1989

We give systolic implementations for three variants of the move-to-front coding algorithm discussed in recent journal articles by Ryabko, Elias, and Bentley, Sleator, Tarjan, and Wei. Our intent is to make a simple highspeed text-compression system for both data transmission and data storage devices. Comparing our implementations with previously-proposed software and hardware devices for these applications, we conclude that our implementations should operate at much higher bandwidth than any known high-compression algorithm, and at higher bandwidth and comparable compression to the Lempel-Ziv-Welch and Smith-Storer hardware.Permission to copy without fee all or part Of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee a n d / o r specific permission.© 1989 A C M 0 -8 9 7 9 1 -3 2 3 -X / 8 9 / 0 0 0 6 / 0 2 8 3 $1.50

“…Like MWP, Rogers-Thomborson code RT [20] uses a back-end coder to transmit frequentlyused codewords with fewer bits than the rarely-used ones. Since RT uses an arithmetic code rather than a static Huffman code, however, it is very slow.…”

Section: Mwp Ofmentioning

confidence: 99%

“…Section 4 lists the compression ratios obtained by a sample V.42bis implementation, as well as the ratios obtained on the same files by the 4.3 bsd Unix "compress" utility, and by state-ofthe-art Ziv-Lempel codes by Miller and Wegman [16], Rogers and Thomborson [20], and Fiala and Greene [13]. My data shows that a well-implemented V.42bis modem will acheive 2.3:l compression on English text, 1.6:l compression on binaries, and 3.2:l compression on source code.…”

Section: Introductionmentioning

confidence: 99%

V.42bis and other Ziv-Lempel variants

[1991] Proceedings. Data Compression Conference

The recently adopted V.42bis standard for data compressing modems is discussed from algorithmic, experimental, practical, and marketing standpoints. V.42bis is a conservative and economically-implementable scheme, one that compresses text about as well as the LempelZiv-Welch algorithm of the Berkeley Unix "compress" utility. Other Ziv-Lempel variants are discussed briefly. Even though better compression ratios can be obtained by these variants, V.42bis is shown to be eminently suitable for implementation on a contemporary modem with an 8-bit microprocessor, 40 Kbytes of RAM, 32 Kbytes of ROM, a 9.6 KBaud V.32 modemmodem connection, and a 19.2 KBaud EIA-232-D modem-terminal connection. I IntroductionIt is now possible to buy a modem with the following characteristics. I will call such a device a V.42bis modem, although I caution the reader that other uses of the V.42bis standard are possible. The implementation and pricing information correspond to a MultiTech MT932EAB, introduced in late 3uly 1990. Six similar products are currently being offered by other manufacturers, according to an informative set of product reviews in PCReaders who are unfamiliar with the communication standards in the list above will find some information in Section 2 below. Should that discussion prove insufficient, I heartily encourage "Research supported by the National Science Foundation, through its Design, Tools and Test Program under grant number MIP 8706139. 1 you to read the PC Magazine reviews [7] and to obtain a copy of Black's excellent compendium and discussion of the V-, EIA-, and X-series standards [5].It is worth noting at the outset that the V.42bis modem's throughput is often limited by its EIA-232-D connection to the terminal or computer, or by the terminal or computer itself. At the 3:l compression ratio observed on some transmissions, a V.42bis modem has a potential throughput of 3200 characters per second. Unfortunately, throughputs of this magnitude are rarely observed in practice, for the reasons listed in Section 2.In Section 3, I describe the data compression method of V.42bis. Briefly, it is a textual substitution scheme based on Ziv and Lempe17s second algorithm [27, 121. V.42bis incorporates Welch's coding scheme [25, 111, as well as Miller and Wegman's (and, independently, Storer's) idea for incremental modification of a full dictionary using a least-recently-used (LRU) heuristic [16,17,22,8,23, 151. V.42bis also has an efficient "cleartext" escape mechanism, differing in implementation but not in spirit from that proposed by Fiala and Greene [13].Section 4 lists the compression ratios obtained by a sample V.42bis implementation, as well as the ratios obtained on the same files by the 4.3 bsd Unix "compress" utility, and by state-ofthe-art Ziv-Lempel codes by Miller and Wegman [16], Rogers and Thomborson [20], and Fiala and Greene [13]. My data shows that a well-implemented V.42bis modem will acheive 2.3:l compression on English text, 1.6:l compression on binaries, and 3.2:l compression on source code...