Data Compression Using Adaptive Coding and Partial String Matching

Abstract

The recently developed technique of arithmetic coding, in conjunction with a Markov model of the source, is a powerful method of data compression in situations where a linear treatment is inappropriate. Adaptive coding allows the model to be constructed dynamically by both encoder and decoder during the course of the transmission, and has been shown to incur a smaller coding overhead than explicit transmission of the model's statistics. But there is a basic conflict between the desire to use high-order Markov models and the need to have them formed quickly as the initial part of the message is sent. This paper describes how the conflict can be resolved with partial string matching, and reports experimental results which show that mixed-case English text can be coded in as little as 2.2 bits/ character with no prior knowledge of the source.

A Two-Level Structure for Compressing Aligned Bitexts

“…In [14] text compression methods are considered for its extension to bitext compression considering exact correspondences between two words, and synonymy relationships between the words in both texts (as given by a thesaurus). These parallel predictions are then combined with PPM [3] ones. The weighting of both models are carefully tuned improving PPM compression ratios on separate texts.…”

“…-a Spanish-Catalan (es-ca) bitext from El Periódico de Catalunya, 2 a daily newspaper published both in Catalan and Spanish; -a Spanish-Galician (es-gl) bitext from Diario Oficial de Galicia, 3 the bulletin of the Government of Galicia, published both in Galician and Spanish; and -bitexts for German-English (de-en), Spanish-English (es-en) and French-English (fr-en) from the European Parliament Proceedings Parallel Corpus [8].…”

Fast hands-free writing by gaze direction

“…Each user took dictation from Jane Austen's Emma in five-minute sessions. The language model (PPMD5) predicts the next character given the previous five characters; 6,7 it was trained on passages from Emma not included in the dictation. Right panels, the two experts took dictation using the same eyetracker to control the WiViK on-screen keyboard (a standard qwerty keyboard) with the word-completion buttons enabled.…”

Compressing Web Geodata for Real-Time Environmental Applications

“…The leading data representation method for compression purposes is Huffman coding, which forms the basis of most subsequent approaches [20]. Compression techniques can be broadly classified in four major categories: derivatives of Lempel-Ziv-Welch [23], approaches based on statistical model prediction [4], on characters permutations [1], and on arithmetic coding [16]. In the case of DEM data, the seminal work [12] advocated the possibility of reducing data size with an initial data simplification stage, followed by compression with Huffman coding.…”

“…Prediction by Partial Matching (PPM) [4] algorithms consider the correlation between values (which, for example, could be linearly growing). They use N past values to predict the next one, trying to find the best relationship.…”